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Overview 


Introduction 

Thi s report presents implementation and early impact results from a random assignment evaluation 
of the Young Adult Internship Program (YA1P), a subsidized employment program for young 
people in New York City who have become disconnected from school and work. Operated by 
various provider agencies, YAIP offers disconnected youth between the ages of 16 and 24 a 
temporary paid internship, as well as various support services. 

The YAIP evaluation is part of the larger Subsidized and Transitional Employment Demonstration, 
sponsored by the Administration for Children and Families in the U.S. Department of Health and 
Human Services. From July 2013 to March 2014, researchers assigned nearly 2,700 young people at 
random to either a program group, which was offered YAIP services, or to a control group, which 
was not offered those services. The YAIP evaluation will measure outcomes for both groups over 
time to assess whether YAIP services led to better outcomes for the program group compared with 
those of the control group. 

This report is the first of two focused on the YAIP evaluation. It provides a detailed description of 
the YAIP model, assesses its implementation, and examines whether the program improved young 
people’s outcomes during the first year after study enrollment. Overall, the implementation study 
concluded that the YAIP program was well implemented across provider agencies and that partici- 
pation rates were high. An analysis of youth outcomes indicates that program group members were 
more likely than control group members to receive employment and other types of support. In part 
due to the offer of a paid internship, the program group was also more likely to work and had higher 
earnings in the year following random assignment. Additional follow-up will be necessary to assess 
YAIP’s longer-term effects on employment and earnings, as well as other key outcomes. 

Primary Research Questions 

This study seeks to answer the following research questions: 

• How was YAIP designed and operated? 

• What impact did YAIP have on employment and earnings, education and training, and well- 
being relative to what would have happened in the absence of the program? Did YAIP appear to 
be more effective for certain subgroups of young people? 

• To what extent do YAIP’s costs differ from those expended on behalf of individuals randomly 
assigned to a control group that could not receive YAIP program services? How does this cost 
differential relate to the benefits associated with program impacts, if any? 

Purpose 

For many young people, the time between one’s late teenage years and early twenties encompasses 
several important milestones, including graduating from high school, attending college, entering the 
workforce, and beginning to establish economic independence. However, 12.3 percent of young 



people in the United States between the ages of 16 and 24 — 4.9 million people in total — are 
neither in school nor working. These “disconnected” or “opportunity” youth face serious challenges 
to achieving labor market success and self-sufficiency in adulthood. 

YA1P is intended to help reengage youth who have fallen off track, thereby reducing their risk of 
long-term economic hardship. MDRC is conducting a random assignment evaluation of YAIP to 
determine whether the program makes a difference in the lives of the young people it serves. 

Key Findings 

Findings from the report include the following: 

• Overall, YAIP was well implemented. The program was delivered very similarly across 
providers and with a high degree of fidelity to the program model as designed. Participation 
rates were high: Over three-fourths of young people assigned to the program group worked in a 
subsidized internship and 86 percent of those youth completed the internship. 

• Program group members were more likely than control group members to report receiving 
employment support, as well as advice or support and mentorship from staff members at an 
agency or organization. However, substantial numbers of control group members also received 
help in these areas. 

• The program group was more likely than the control group to have worked during the year 
following random assignment, but the employment rates of the two groups converged during the 
quarters after the YAIP internships ended. The program group also had higher earnings than the 
control group. While largest during the time when program group members were working in 
paid internships, these earnings impacts persisted throughout the follow-up period, suggesting 
that program group members may have obtained better jobs than control group members. 

Methods 

The evaluation includes an implementation study, an impact study, and a benefit-cost analysis. This 
report presents implementation and early impact findings (after one year). Benefit-cost findings and 
longer-term impact findings (after 30 months) will be presented in a future report. 

The implementation study describes YAIP’s design and how the program ultimately operated. Key 
data sources for the implementation study include staff interviews, observations, and participation 
data. The implementation section of this report integrates qualitative and quantitative data from these 
sources to create a coherent picture of the implementation of the program. 

The impact study uses a randomized controlled trial design in which individuals eligible for and 
interested in YAIP were randomly assigned to either a program group, which was offered YAIP 
services, or to a control group, which was not offered those services. The study will evaluate impacts 
on employment and earnings, education and training, and well-being, among other areas. Data 
sources for the impact study include administrative records on wages and postsecondaiy enrollment, 
subsidized employment payroll records, and surveys conducted approximately 4, 12, and 30 months 
after participants entered the study. 
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Executive Summary 


For many young people, the time between one’s late teenage years and early twenties encom- 
passes several important milestones, including graduating from high school, attending college, 
entering the workforce, and beginning to establish economic independence. For some, however, 
staying engaged in education or employment during the transition to adulthood can be difficult: 
12.3 percent of young people in the United States between the ages of 16 and 24 — 4.9 million 
young people in total — are neither in school nor working. This group is commonly referred to 
as “disconnected youth” or “opportunity youth.” As a result of low levels of educational 
attainment and limited work experience, disconnected youth face serious challenges to achiev- 
ing labor market success and self-sufficiency in adulthood. 1 

The Young Adult Internship Program (YAIP) is intended to help reengage youth who 
have fallen off track, thereby reducing their risk of long-term economic hardship. The New 
York City Center for Economic Opportunity (CEO) and the New York City Department of 
Youth and Community Development (DYCD) oversee the program and community-based 
provider organizations throughout the city deliver it. YAIP offers youth various services, 
including job-readiness workshops and activities; individual support, counseling, and assess- 
ments; case management; and follow-up services. However, the central program component is a 
10- to 12-week paid internship. 

This report presents interim findings from a random assignment evaluation of YAIP, 
which is studying the program’s implementation and “impacts,” or the difference the program 
makes in the lives of the youth it serves. The evaluation also includes a benefit-cost study. 
Young adults who enrolled in the evaluation include low-income New York City residents 
between 16 and 24 years of age who were neither working nor in school. The YAIP evaluation 
is part of a larger evaluation funded by the Administration for Children and Families in the U.S. 
Department of Health and Human Services (HHS) called the Subsidized and Transitional 
Employment Demonstration (STED), which is testing various subsidized employment strategies 
in several cities across the country. MDRC is conducting the STED Evaluation, along with its 
research partners MEF Associates, Decision Information Resources, and Branch Associates. 

Background 

In early adulthood, it is important to gain skills and experience through education, training, and 
employment in order to establish a solid foundation for future success. However, many young 


'Sarah Burd-Sharps and Kristen Lewis, Promising Gains, Persistent Gaps: Youth Disconnection in Amer- 
ica (New York: Measure of America of the Social Science Research Council, 2017). 
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people in the United States are neither enrolled in school nor participating in the labor market. 
The detachment of these young people from society’s larger structures of school and work 
poses serious costs to their future well-being, their communities, and to the country as a whole. 2 

There are a variety of reasons why young people may become disconnected, including a 
lack of support and guidance from adults; family obligations; issues with school safety, school 
policies, and peer influences; and the challenges of young parenthood. 3 Disconnected youth 
may also simply have trouble finding work. In recent years, concern about at-risk, out-of- 
school, and out-of-work young people has grown among policymakers, service providers, and 
other key stakeholders. This concern has generated new policies and initiatives to better serve 
and reconnect this population to education, training, and employment. One example of these 
efforts is the Workforce Innovation and Opportunity Act (WIOA), which President Obama 
signed into law in 2014. WIOA places increased emphasis on providing employment and 
training services to disconnected young people who are out of school. 

Disconnected youth are a heterogeneous group in terms of the causes of their discon- 
nection, their level of disadvantage, their educational backgrounds, and the length of their 
disconnection. Because of this heterogeneity, appropriate service models vary considerably in 
tenns of the subpopulations they target, the services they provide, their level of intensity, and 
their underlying theories for how best to recomiect young people to school, training, or work. 
Programs targeting disconnected young people tend to offer different combinations of educa- 
tional support, job skills training, paid or subsidized employment, case management, and other 
services. A number of these programs have been rigorously evaluated, with some showing 
positive, statistically significant effects, primarily on employment and earnings. 4 YAIP is 
unique among disconnected youth programs in a few respects: it is a relatively simple model 
focused on work experience; it targets a more job-ready subset of disconnected youth (this point 
will be discussed in greater detail below); and it operates at large scale. 5 Thus, learning about 
the effectiveness of YAIP will contribute to the existing research evidence regarding what 
works to reengage low-income disconnected young people in education and work. 


2 Burd-Sharps and Lewis (2017). 

3 Michelle Hynes, Don ’t Call Them Dropouts: Understanding the Experiences of Young People Wlw 
Leave High School Before Graduation (Washington, DC: America’s Promise Alliance, 2014). 

4 Louisa Treskon, Wluit Works for Disconnected Young People: A Scan of the Evidence (New York: 
MDRC, 2016); Farhana Hossain and Dan Bloom, Towards a Better Future: Evidence on Improving Employ- 
ment Outcomes for Disadvantaged Youth in the United States (New York: MDRC, 2015). 

5 ln fiscal year 2015 (July 1, 2014, through June 30, 2015), 18 YAIP providers across New York City 
served 1,821 young people. 
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The Young Adult Internship Program (YAIP) 

Introduced in 2007, YAIP is a workforce development program designed for young New 
Yorkers living in poverty who are neither in school nor working, but who are believed to have 
the potential to benefit from a relatively brief, non-intensive intervention. In theory, the YAIP 
target population is not so disadvantaged that they need extensive wraparound services, but 
disadvantaged enough to require the support of a program to acquire the skills and experience 
needed to improve their labor market prospects and give them a “jump start” back into produc- 
tive activity. 

YAIP is a multiphase program that enrolls youth in cohorts, with a new cohort starting 
every four months; participants in a particular cohort move through the program together. Each 
community-based YAIP provider is responsible for enrolling and serving a portion of the full 
cohort, usually about 30 young adults each. The program’s three phases are as follows: 

• Phase 1: The first 2 to 4 weeks of the program (duration varies by provider) 
are referred to as the orientation phase, wherein youth are expected to attend 
daily workshops facilitated by program staff at provider offices. Youth are 
paid minimum wage for 25 hours per week, 6 and workshops are typically 
five hours per day. The goals of orientation are to prepare participants for the 
workplace by providing various job-readiness and personal development ac- 
tivities, to lay a foundation for cohort cohesion using icebreakers and group 
activities, and to match participants’ interests and skills with an available and 
appropriate internship. 

• Phase 2: During the 10 to 12 weeks of this phase, youth are expected to 
work 20 hours a week in their internship placement and continue to earn 
minimum wage. Their earnings are fully subsidized. The goals of internship 
placements vary based on the particular needs of young people, but generally 
include work experience, development of soft or hard skills, career explora- 
tion, and potential transition from a subsidized internship to a pennanent, un- 
subsidized position. Once a week, youth are required to return to the provider 
offices to attend five-hour educational workshops, for which they are also 
paid minimum wage. These workshops cover topics including job readiness, 
healthy living, money and time management, and conflict resolution. 


6 The minimum wage increased from $7.25 to $8 per hour in New York during the study period. As of this 
report’s publication, the minimum wage in New York City was $11 per hour for employers of 11 or more 
employees and $10.50 for employers of 10 or fewer employees. 
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• Phase 3: The nine months following youths’ completion of their internship is 
the follow-up phase of YAIP. During this time, providers are expected to 
help participants secure and maintain an “outcome placement.” Outcome 
placements include participation in unsubsidized employment, education, 
training, or the military. Providers also offer support services during this 
phase, including housing assistance, counseling, and transportation assis- 
tance, among other types of support. 

The YAIP Evaluation 

Earlier evaluations of subsidized employment programs targeting groups who struggle most in 
the labor market, including disconnected youth, the formerly incarcerated, welfare recipients, 
noncustodial parents, and others, yielded mixed results. Designed to advance the field’s under- 
standing of subsidized employment, STED is studying eight different subsidized employment 
program models in six cities across the country. Each model, including YAIP, is being evaluat- 
ed independently in a randomized controlled trial. 

The random assignment design of the YAIP evaluation is generally considered to be the 
most rigorous method of evaluating large-scale social service programs. This research design 
involves a lottery-like process that places individuals into either a program group, which is 
offered the services being tested, or into a control group, which is not offered those services. As 
a result of the random assignment process, these two groups will be comparable on both 
measured and umneasured characteristics at the time of study enrollment. The evaluation 
includes 12 independent YAIP providers that delivered the program at 13 different locations 
across Brooklyn, the Bronx, Manhattan, and Queens. YAIP providers targeted individuals 
between 16 and 24 years of age who were neither in school nor working, among other eligibility 
criteria. 


The YAIP evaluation enrolled a total of 2,678 young people in three consecutive co- 
horts, beginning with the July 2013 cohort, followed by the November 2013 cohort, and 
concluding with the March 2014 cohort. The research team randomly assigned 60 percent of the 
sample to the program group and 40 percent to the control group: 

• The program group. The 1,638 individuals who were randomly assigned to this group 
were offered YAIP program services, including a paid internship, job-readiness train- 
ing, case management, and follow-up services. 

• The control group. The 1,040 individuals who were randomly assigned to this group 
were not offered YAIP program services, but were able to access other services that 
were available in the community, including other non-YAIP services offered at YAIP 
provider agencies. 
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By measuring outcomes for both groups over time, it is possible to assess whether 
YAIP services led to better outcomes for the program group than what would have happened in 
the absence of the program, as represented by the control group. Indeed, any statistically 
significant differences that emerge between the two randomly assigned groups provide a 
reliable estimate of the program’s effects, or “impacts.” 

The YAIP evaluation has three components: an implementation study, an impact study, 
and a benefit-cost study. The primary goal of the implementation study is to describe the design 
of the YAIP program and how it operates. As noted above, the impact study addresses the 
question of whether YAIP improves key outcomes of interest for disconnected youth, including 
employment and earnings, education and training, psychosocial outcomes, economic and 
personal well-being, and involvement with the criminal justice system. Finally, the benefit-cost 
study will compare YAIP’s costs with its benefits. This report focuses on the results of the 
implementation study and early findings from the impact study (based on one year of follow-up 
data). Final results from the impact study (based on 30 months of follow-up data) and benefit- 
cost analysis will be included in a future report, expected in 2018. 

Implementation of YAIP 

The research team assessed YAIP’s implementation using several different data sources, 
including (but not limited to) interviews with key provider staff as well as YAIP management 
staff from DYCD and CEO, internship worksite observations, analysis of data collected from all 
young people when they enrolled in the study, and program participation data from the YAIP 
management infonnation system. 

To be eligible for YAIP, applicants must be New York City residents, between the ages 
of 16 and 24, and eligible to work in the United States; must neither be in school nor working; 
and cannot have been enrolled in school during the prior semester. In addition, all YAIP 
applicants are required to complete a multistage intake process designed to screen for appropri- 
ate participants — that is, the most job-ready discomiected youth, YAIP’s target population. 7 
The intake process includes a reading level assessment, a self-assessment fonn, and an inter- 
view. The research team randomly assigned applicants who were detennined eligible at the 
conclusion of the assessment and intake process to one of the two research groups that comprise 
the YAIP study sample. 


’Provider staff assessed job-readiness based on several criteria, including motivation level, stability of 
housing and child care arrangements, mental health, reading level, and availability of social support, among 
other factors. 
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The YAIP sample was 21 years of age, on average, at the time of study enrollment and 
is divided about evenly between men and women. The vast majority of sample members are 
Black, non-Hispanic (58 percent) or Hispanic (36 percent). Sixty-one percent of the sample had 
earned a high school diploma or equivalency certificate and nearly three-fourths of the sample 
had previous work experience (though the vast majority of this latter group had not worked in 
any job for three months or longer). The median length of time since study youth were last in 
school, enrolled in a high school equivalency program, or working was nine months. Only 
about 4 percent of the sample reported experiencing unstable housing. Finally, over one-fourth 
of the sample received some form of public assistance and, in addition to their limited schooling 
and employment histories, 42 percent of sample members faced at least one other significant 
barrier to employment, including limited literacy or math skills, housing instability, parenting 
responsibilities, a disability, or a criminal record. 

• Young people in the study sample compare favorably with the broader 
population of disconnected youth in New York City on various socioeco- 
nomic indicators, suggesting that YAIP providers successfully targeted a 
more job-ready subset of disconnected youth. 

Compared with local and national populations of disconnected youth, YAIP sample 
members were more likely to have a high school credential and previous work experience and 
less likely to receive public benefits or to live in unstable housing situations at the time of study 
enrollment. Thus, while the YAIP sample still faced serious challenges, providers successfully 
identified and enrolled the subset of disconnected youth thought best able to benefit from the 
program. 

• Overall, the implementation of YAIP aligns with the program model 
and is consistent across providers. 

DYCD allows providers some flexibility in how they operate the program, including in 
staffing decisions and the duration of program phases. Despite this flexibility and the range of 
different organizations that offered the program — each with its own focus and varying set of 
resources — the research team found that, overall, providers delivered the YAIP model very 
similarly and with a high degree of fidelity to the intended program model, as described above. 

Nevertheless, program implementation faced a number of challenges. Provider staff re- 
ported difficulties with navigating the complex and lengthy recruitment and assessment process; 
contending with YAIP’s overlapping program cycles, which required them to juggle varying 
responsibilities for cohorts in different phases of the program simultaneously; the challenges 
inherent in working with a disadvantaged group of young people, some of whom require a great 
deal of support even outside of regular working hours; and meeting DYCD’s oversight and 
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performance expectations. Program managers reported that these difficulties left some staff 
members vulnerable to burnout, contributing to frequent staff turnover. 

Based on data from YAIP’s management information system, participation in YAIP 
was high. As shown in Figure ES.l, which depicts participation among 100 typical program 
group members, over 80 percent of the program group attended the paid orientation phase, 
nearly all of whom completed it. During this phase, provider staff facilitate daily group sessions 
at program offices to help meet the first two goals of orientation, which are preparing partici- 
pants to succeed in the workplace and laying a foundation for cohort cohesion. Staff achieve the 
third goal of orientation, matching youth to internship sites, through a collaborative process in 
which staff and participants meet one-on-one to discuss participants’ strengths, interests, and 
goals. Provider staff consider all of these dimensions, as well as the participants’ personalities 
and the internship site’s work environment and culture, in matching participants to internship 
placements. 

• More than three-fourths of the program group worked in a DYCD- 
subsidized internship. Among this group, 86 percent completed their in- 
ternship. 

Internship placements are at a wide variety of worksites, running the gamut from the of- 
fices of local politicians to commercial drugstores, social service nonprofit agencies, clothing 
retail chains, and a local radio station. Regardless of worksite, most young people are assigned 
tasks that involve clerical work, customer service, or maintenance. Internships promoted the 
program goals in various ways, such as providing participants with employment and income, 
exposing them to new workplace experiences, and bolstering their resumes. Participants who 
worked in a subsidized internship averaged 9.2 weeks in their placement, close to the full 
internship length of 10 to 12 weeks. Additionally, during the internship phase, this group spent 
an average of about 22 hours per week in their internships and in the mandatory educational 
workshops, close to the expected 25 hours per week. The educational workshops, held once a 
week at provider offices, offered an opportunity for youth to stay in touch with members of 
their cohort, check in with case managers, and receive ongoing job-readiness skills development 
while they worked in their internships. Despite participants receiving pay for time spent in 
educational workshops, providers reported that attendance was often an issue. 

Case managers begin “outcome planning” with their participants about halfway through 
the program, aiming to engage participants in one of the four DYCD-endorsed post-program 
placements by the end of the internship period. These preferred placements are unsubsidized 
employment, education, advanced training, and military service. Based on participation data, a 
little more than half of program group members who worked in an internship were engaged in 
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Figure ES.l 

YAIP Participation Among 100 Typical Program Group Members 


Program group members 


100 


Attended orientation 


Completed orientation 


79 


Worked in subsidized internship 


Completed subsidized internship 


77 


66 


SOURCE: MDRC calculations using data from the YAIP management information system. 

NOTES: The data presented in this figure are based on the 1,638 program group members in the YAIP study. 


one of these outcome placements at the conclusion of their internship, the vast majority of them 
in unsubsidized employment, with education a distant second. Participants pursuing advanced 
training were far rarer, and those enlisting in military service were negligible. 

Whether or not participants were engaged in a placement at the conclusion of their in- 
ternship, provider staff continued to offer them follow-up support services over the next nine 
months. These nine months make up Phase 3 of YAIP, or the follow-up phase, when providers 
focus on helping participants secure or sustain a placement. This follow-up period is the longest 
yet least intensive phase of the program cycle, when staff work with participants mainly on an 
individual basis. During this phase, provider staff try to maintain contact with the young people 
to assist them with job leads, verily their outcome placements, help connect them to needed 
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services, and provide other case management and counseling support. Eighty-six percent of 
participants who worked in an internship received case management during this phase, and had 
an average of five contacts with YAIP staff. 8 The proportion of participants engaged in a 
DYCD-approved placement at the nine-month mark is very similar to the proportion engaged in 
a placement at the end of the internship period, as is the distribution of participants across the 
four preferred placements. The proportion of youth in verified placements fell short of DYCD’s 
performance benchmarks at both junctures. 

YAIP was developed as a “light-touch” intervention to help reengage a subset of the 
most job-ready disconnected youth based on the premise that this group would not require 
intensive services to get back on track. However, many involved with the YAIP program, 
including provider staff, supervisors at internship sites, and participants themselves, disagreed 
with this premise and believed that participants require both a higher level of support and a 
longer intervention to improve their educational and labor market outcomes. 

Early Impacts of YAIP 

The impact study relies on data from three key sources: employment and earnings data from the 
National Directory of New Hires (NDNH), 4- and 12-month follow-up survey data, and 
postsecondary school enrollment data from the National Student Clearinghouse. 9 For this report, 
follow-up data for just over one year after random assigmnent was available to assess differ- 
ences between program and control group members. 

While control group members could not enroll in YAIP, they were able to access other 
services that were available in the community. In New York City, services for disadvantaged 
young people are plentiful. Using survey data, the research team assessed the extent to which 
the offer of the YAIP program increased the services received by the program group over and 
above what the control group received. Without a meaningful service differential, statistically 
significant impacts on youth outcomes are unlikely. 

• Program group members were more likely than control group members 
to report receiving employment support as well as advice or support and 
mentorship from staff members at an agency or organization. 


S A contact was counted as any time a case manager was able to successfully reach a participant, including 
phone, social media, and in-person contact. 

9 Maintained by the federal Office of Child Support Enforcement, the National Directory of New Hires 
contains quarterly earnings data collected by state workforce agencies on jobs covered by unemployment 
insurance. 
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Results show that the program group was significantly more likely than the control 
group to have been employed in a paid internship (60 percent versus 8 percent). 10 They were 
also more likely to have received help finding or keeping a job (85 percent versus 53 percent), 
which includes participation in activities such as job search, job readiness, and career planning, 
as well as financial support for job-related costs. Additionally, program group members were 
significantly more likely than control group members to have received advice or support (70 
percent versus 46 percent) and mentoring (62 percent versus 35 percent) from staff at an agency 
or organization. In sum, while many control group members received services, program group 
members received substantially more services in the key areas on which YAIP focuses. 

• YAIP increased employment in the year following random assignment, 
but this program effect dissipated by the end of the follow-up period. 
Program group members had higher earnings than control group mem- 
bers throughout the follow-up period. 

As shown in the top panel of Figure ES.2, which is based on NDNH data, program 
group members were more likely than control group members to work in the year following 
random assignment, but the quarterly employment rates of the two research groups converged 
shortly after the YAIP subsidized internships ended. 11 Turning to the bottom panel of the figure, 
program group members also had higher earnings than control group members, earning over 
$3,433 more than control group members over the course of the year (quarter 0 through quarter 
3). Statistically significant earnings impacts were observed in every quarter of the year follow- 
ing random assignment, although the size of the impact peaked during the first quarter after 
random assignment (when participation in YAIP internships was highest) and dropped steeply 
thereafter, remaining relatively consistent in the later quarters. The differences in earnings that 
persisted in the later quarters, along with data from the 12-month survey concerning partici- 
pants’ current employment, suggest that program group members may have been able to secure 
better jobs (that is, more permanent positions with full-time hours) than control group members. 
This possible program impact will be more fully examined later, when additional follow-up data 
past 12 months is available. 

• Program and control group members had si mi lar outcomes during the 
first year of follow-up in other key domains. 


10 These figures are based on data from the four-month follow-up survey. It is likely that the survey ques- 
tion regarding paid internships was subject to some reporting error, which may explain the discrepancy 
between these program group internship participation rates and those based on management information 
system data. 

1 'NDNH data are reported quarterly; thus, employment and earnings in this report cover the quarter during 
which random assignment occurred (quarter 0), as well as the four quarters following the quarter of random 
assignment, for a total of five quarters. These data include participation in paid YAIP activities. 
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Average earnings ($) Percentage employed 


Figure ES.2 

Employment and Earnings Over Time 
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Figure ES.2 (continued) 


SOURCES: MDRC calculations based on quarterly wage data from the National Directory of New Hires and 
YAIP management information system subsidized earnings records. 


NOTES: Results in this table are regression-adjusted, controlling for pre -random assignment characteristics. 
Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 


Aside from the differences observed in the employment and earnings domain, program 
and control group members had similar outcomes over the first year of follow-up in the other 
domains where impacts were assessed. These domains include education and training, psycho- 
social outcomes, economic and personal well-being, and criminal involvement. 

However, results of the four-month survey, which the research team administered at a 
point in time when many program group members were working in their paid internships, 
showed some positive effects on young people’s happiness and perception of obstacles to 
achieving their goals. This result suggests that YAIP led to some level of improved well-being 
for young people while they were working in subsidized internships, even if these improve- 
ments do not appear to have persisted after the internships ended. 

Next Steps 

The YAIP evaluation is part of a larger effort to understand how best to help young people who 
have become untethered from the worlds of school and work to reengage in productive activity. 
Findings from the implementation study indicate that YAIP is a well-implemented program, 
operates similarly across providers with a high degree of fidelity to the program model, and 
serves a large swath ofNew York City’s more job-ready disconnected youth. Rates of participa- 
tion are high, a notable finding considering that many youth programs struggle to keep young 
people engaged in their services. Whether YAIP is having its ultimate intended effects of 
improving participants’ labor market prospects and reducing their risk of long-term economic 
hardship remains an open question. The current report presents only 12-month impacts of the 
program. It is too early to draw any firm conclusions about whether YAIP will improve em- 
ployment outcomes or outcomes in other domains in the longer term. Final impact results, with 
a longer-term follow-up of 30 months, will be presented in a later report, as will the results of a 
benefit-cost study. 
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Chapter 1 

Introduction 


For many young people, the time between one’s late teenage years and early twenties encom- 
passes several important milestones, including graduating from high school, attending college, 
entering the workforce, and beginning to establish economic independence. For some, however, 
staying engaged in education or employment during the transition to adulthood can be difficult: 
12.3 percent of young people in the United States between the ages of 16 and 24 — 4.9 million 
young people in total — are neither in school nor working. These young people are commonly 
referred to as “disconnected youth” or “opportunity youth.” As a result of low levels of educa- 
tional attainment and limited work experience, disconnected youth face serious challenges to 
achieving labor market success and self-sufficiency in adulthood. 1 

The Young Adult Internship Program (YAIP) is intended to help reengage young peo- 
ple who have fallen off track, thereby reducing their risk of long-term economic hardship. The 
New York City Center for Economic Opportunity (CEO) and the New York City Department 
of Youth and Community Development (DYCD) oversee the program and community-based 
provider organizations throughout the city deliver it. YAIP offers youth various services, 
including job-readiness workshops and activities; individual support, counseling, and assess- 
ments; case management; and follow-up services. However, the central program offering is a 
10- to 12-week paid internship. 

This report presents interim findings from a random assignment evaluation of YAIP, 
which is studying the program’s implementation and “impacts,” or the difference the program 
makes in the lives of the youth it serves. The evaluation also includes a benefit-cost study. 
Young adults who enrolled in the evaluation include low-income New York City residents 
between 16 and 24 years of age who were neither working nor in school. The YAIP evaluation 
is part of a larger evaluation funded by the Administration for Children and Families in the U.S. 
Department of Health and Human Services (HHS) called the Subsidized and Transitional 
Employment Demonstration (STED), which is testing various subsidized employment strategies 
in several cities across the country. MDRC is conducting the STED evaluation, along with its 
research partners MEF Associates, Decision Information Resources, and Branch Associates. 


'Burd-Sharps and Lewis (2017). 
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Background and Policy Context 

In early adulthood, it is important to gain skills and experience through education, training, and 
employment in order to establish a solid foundation for future success. However, many young 
people in the United States are neither enrolled in school nor participating in the labor market. 
The detachment of these young people from society’s larger structures of school and work 
poses serious costs to their future well-being, their communities, and to the country as a whole. 2 
Low levels of education and limited work experience can have long-term effects on earnings, 
self-sufficiency, health, and family formation. Youth disconnection is likely to affect society via 
lost productivity and tax contributions, increased dependence on public assistance, and higher 
rates of criminal activity. 3 

There are a variety of reasons why young people may become disconnected, including a 
lack of support and guidance from adults; family obligations; issues with school safety, school 
policies, and peer influences; and the challenges of young parenthood. 4 Disconnected youth 
may also simply have trouble finding work. Young jobseekers have had a particularly challeng- 
ing time in recent years following the collapse of the youth labor market during the Great 
Recession and its slow subsequent recovery. 5 Youth disconnection often mirrors adult discon- 
nection. As noted in a recent report on the subject by the Social Science Research Council, 
household poverty rates and the employment and educational status of adults in a community 
strongly correlate with youth disconnection in that community. 6 Reflecting the higher poverty 
rates, lower educational attainment, and higher unemployment that characterize many minority 
communities, Black and Hispanic youth are overrepresented among the larger population of 
disconnected youth. 7 

In recent years, concern about at-risk, out-of-school, and out-of-work young people has 
grown among policymakers, service providers, and other key stakeholders. This concern has 
generated new policies and initiatives to better serve and reconnect this population to education, 
training, and employment. One example of these efforts is the Workforce Innovation and 
Opportunity Act (WIOA), which President Obama signed into law in 2014. Relative to its 
predecessor (the Workforce Investment Act), WIOA places increased emphasis on providing 
employment and training services to disconnected young people who are out of school and 
raises the upper age limit for these services from 21 to 24. The next section describes some of 


2 Burd-Sharps and Lewis (2017). 

3 Belfield, Levin, and Rosen (2012). 

4 Hynes (2014). 

5 Sum, Khatiwada, Trubskyy, and Palma (2014); Ayres (2013). 

6 Burd-Sharps and Lewis (2017). 

7 Belfield, Levin, and Rosen (2012). 
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the existing service models targeting the needs of this population, as well as the research 
evidence regarding the effectiveness of these models. 


Service Models for Disconnected Youth and Research Evidence 
Regarding Their Effectiveness 

Disconnected youth are a heterogeneous group, in tenns of the causes of their disconnection, 
their level of disadvantage, their educational backgrounds, and the length of their disconnection. 
With respect to the length of disconnection, it is important to note that disconnection is not a 
static condition. While some disconnected young people are considered “chronic,” or have not 
been enrolled in school or attached to the labor market since the age of 16, many others are 
“underattached,” or, despite some education or work experience, have not maintained consistent 
ties to either school or the labor market and may cycle in and out of disconnection. 8 Still others 
may encounter temporary setbacks that leave them disengaged from school and work for some 
period of time before they recover and reengage in consistent productive activity. 

In response to this heterogeneity, service models for these young adults vary considera- 
bly in tenns of the subpopulations they target, the services they provide, their level of intensity, 
and their underlying theories for how best to reconnect young people to school, training, or 
work. Programs targeting disconnected young people tend to offer different combinations of 
educational support, job skills training, paid or subsidized employment, case management, and 
other services. A number of these programs have been rigorously evaluated, with some showing 
positive, statistically significant effects, primarily on employment and earnings. This section 
describes several interventions that serve disconnected youth. 

The National Guard Youth ChalleNGe program is an intensive, residential program that 
serves high school dropouts ages 16 to 18 who are drug free and not heavily involved with the 
criminal justice system. The program emphasizes positive youth development and includes 
education, community service, mentoring, and other components in a quasi-military setting. A 
random assignment study conducted by MDRC found that ChalleNGe increased rates of 
General Educational Development (GED) certificate or high school diploma receipt and college 
credit receipt, as well as employment rates and earnings. 9 

Another example of a service model for disconnected young people is New York City’s 
Out-of-School-Youth Program (OSY), also overseen by DYCD. OSY is a yearlong program for 
low-income disconnected youth between the ages of 16 and 24. It provides occupational skills 


8 Belfield, Levin, and Rosen (2012). 

9 Millenky, Bloom, Muller-Ravett, and Broadus (2011). 
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training in several industries, including construction, food service, tourism, health care, and 
retail. In addition, the program offers assistance with job and college placement, GED prepara- 
tion, and support services. 10 OSY has not yet been rigorously evaluated. 

Year Up, a training- and employment- focused model, operates in 18 cities and serves 
participants ages 1 8 to 24 who have a high school diploma or equivalency certificate. It offers 
participants six months of training in information technology and finance, followed by a six- 
month internship. Participants receive a stipend tied to a perfonnance contract during both the 
training and internship phases. Additionally, staff advisers help young people with any issues 
they may encounter, and each participant is paired with a mentor. A small random assignment 
evaluation found that members of the Year Up group had higher earnings in the second year 
after random assigmnent and were more likely to be working full-time in their current or most 
recent job compared with members of a control group. 11 At the three-year mark, the program 
group had higher earnings than the control group, mostly as the result of higher wages. Howev- 
er, the program group was less likely to be attending college than the control group. The three- 
year follow-up findings should be interpreted with caution, as the study had a short embargo 
period (the period during which control group members were prevented from joining the 
program); nearly a third of control group members ultimately ended up participating in Year 
Up. 12 Efforts to learn more about the effectiveness of Year Up are under way as part of the 
Administration for Children and Families’ Pathways for Advancing Careers and Education 
study; early impact results are expected in 2017. 

Operating across the United States, Job Corps is primarily a residential program for dis- 
advantaged young people ages 16 to 24. It combines GED or other high school equivalency 
education with job skills training and case management. Job Corps also helps participants 
transition to employment after the program. 13 Results from a random assigmnent study of Job 
Corps released in 2006 found positive earnings and employment impacts in years three and four 
of the follow-up period, but the impacts faded after that time. The evaluation found that results 
were better for older youth (those ages 20 to 24, compared with those ages 16 to 19). 14 

YouthBuild is a national youth and community development program targeting discon- 
nected youth. A hybrid employment and education or training program, YouthBuild offers low- 
income young people ages 16 to 24 (often those who did not complete high school) a variety of 
services for 6 to 12 months, including financial supports. Participants spend at least half of their 


10 New York City Department of Youth and Community Development (2016c). 
1 Roder and Elliott (2011). 

12 Roder and Elliott (2014). 

13 Job Corps (2013). 

14 Schochet, Burghardt, and McConnell (2006). 
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work week engaged in an array of educational services, including high school equivalency, high 
school diploma, or college preparation. Participants generally spend much of their remaining 
time learning construction skills by building or rehabilitating housing for low-income or 
homeless people. 15 The program also includes a “mental toughness” orientation that screens for 
motivation and other assessments, as well as leadership training, community service, counseling 
and support services, and job placement and follow-up services. MDRC is conducting a random 
assignment evaluation of Youth Build. Interim results show that the program increased partici- 
pation in education and training, receipt of high school equivalency certificates, two-year 
college enrollment, civic engagement, and wages and earnings during the 30 month follow-up 
period. 16 

Project Rise is yet another example of a service model targeting disconnected youth. 
Project Rise serves 18- to 24-year-olds who lack a high school diploma or equivalency certifi- 
cate and have been disconnected from school, work, and training for at least six months. 
Participants enroll in cohorts of 25 to 30 and engage in a 12-month sequence of activities 
including case management, classroom education focused on preparation for a high school 
diploma or equivalency certificate, and a paid part-time internship that is conditional on ade- 
quate attendance in the educational component of the program. Project Rise has been delivered 
in three locations in New York City; one in Newark, New Jersey; and one in Kansas City, 
Missouri. Though not a DYCD program, CEO and DYCD were heavily involved in the 
program’s inception and, as a result, many in the field view Project Rise as an enhanced version 
of the YAIP model that focuses on encouraging young people to earn a high school diploma or 
equivalency certificate in addition to securing employment. MDRC conducted an implementa- 
tion study of Project Rise that assessed how the program operates and how providers engage 
young people in program services. The study’s results offer important lessons for policymakers, 
funders, and program operators; however, the study was not designed with a control or compar- 
ison group and therefore was unable to assess the program’s impacts. 17 

YAIP is unique among disconnected youth programs in a few respects: It is a relatively 
simple model focused on work experience, it targets a more job-ready subset of disconnected 
youth (this point will be discussed in greater detail below), and it operates at large scale. 18 Thus, 
learning about the effectiveness of YAIP will contribute to the existing research evidence 
regarding what works to reengage low-income disconnected young people in education and 
work. 


15 Wiegand et al. (2015). 

16 Miller at al. (2016). 

17 Manno, Yang, and Bangser (2015). 

18 In fiscal year 2015, (July 1, 2014, through June 30, 2015), 18 YAIP providers across New York City 
served 1,821 young people. 
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DYCD, CEO, and the YAIP Model 

Established in 1996 when the City of New York merged its Department of Youth Services and 
its Community Development Agency, DYCD was designed to provide community-based youth 
and family programming. The department is primarily responsible for administering available 
city, state, and federal funds to community-based organizations. 19 DYCD funds a range of 
programs of different types, including after-school, community development, family support, 
literacy, youth services, and youth workforce development programs. 

Launched in 2006, CEO is housed in the New York City Office of the Mayor and is 
tasked with ending the cycle of poverty in the five boroughs through innovative, evidence-based 
programs and policies that build human capital. 20 CEO focuses on working with city agencies 
and partners to create, implement, oversee, monitor, and evaluate a variety of anti-poverty 
programs and policies, with the stated goal of producing objective evidence to inform decisions 
about the expansion or elimination of city programs, both through its internal evaluation team 
and through partnerships with independent program evaluators. DYCD and CEO work together 
to administer YAIP. 

Introduced in 2007, YAIP is a workforce development program designed for young 
New Yorkers living in poverty who are neither in school nor working, but who are believed to 
have the potential to benefit from a relatively brief, non-intensive intervention. YAIP partici- 
pants may or may not have a high school diploma or equivalency certificate (a key difference 
between YAIP and Project Rise, whose participants do not have this credential). In theory, the 
YAIP target population is not so disadvantaged that they need extensive wraparound services, 
but disadvantaged enough to require the support of a program to acquire the skills and experi- 
ence needed to improve their labor market prospects and give them a “jump start” back into 
productive activity; this more job-ready group of discomiected youth has been YAIP’s target 
population since the program’s inception. YAIP is a three-phase program, with the length of 
phases varying by provider: Phase 1 is a 2- to 4-week job-readiness training and internship 
matching period, Phase 2 is a 10- to 12-week paid internship (during which youth attend weekly 
educational workshops), and Phase 3 is a nine-month follow-up period during which staff 
support former interns as they pursue one of the four DYCD-endorsed outcome placements: 
employment, education, training, or the military. Youth are paid minimum wage for 25 hours 
per week during Phases 1 and 2; DYCD fully subsidizes wages. YAIP’s stated overarching goal 
is to increase the educational opportunities, career preparation, labor force participation, wage 


19 New York City Department of Youth and Community Development (2016a); New York City Depart- 
ment of Youth and Community Development (2016b). 

20 New York City Center for Economic Opportunity (2017). 
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earnings, job retention, and level of educational attainment of disconnected youth in New York 
City in an effort to reduce their risk of long-term economic hardship. 21 


The YAIP Evaluation 

Earlier evaluations of subsidized employment programs targeting groups who struggle most in 
the labor market, including disconnected youth, the formerly incarcerated, welfare recipients, 
noncustodial parents, and others, yielded mixed results. Designed to advance the field’s under- 
standing of subsidized employment, STED is studying eight different subsidized employment 
program models in six cities across the country. Each model, including YAIP, is being evaluat- 
ed independently in a randomized controlled trial. 22 

The random assignment design of the YAIP evaluation is generally considered to be the 
most rigorous method of evaluating large-scale social service programs. This research design 
involves a lottery-like process that places individuals into either a program group, which is 
offered the services being tested, or into a control group, which is not offered those services. 
CEO and DYCD selected 12 YAIP providers (of 17 that were operating at the time of study 
enrollment) across Brooklyn, the Bronx, Manhattan, and Queens to participate in the STED 
evaluation. 23 Providers targeted individuals between 16 and 24 years of age who were neither in 
school nor working. 24 Other key eligibility requirements included a mandatory score of at least a 
sixth-grade reading level on the Test for Adult Basic Education (TABE), no postsecondary 
degree, and a semester-long waiting period for young adults who had recently graduated from 
high school or dropped out of high school or college. (In theory, this waiting period gives these 
young adults time to reengage in school or work on their own.) Additionally, providers used a 
multistage intake process, which included a self-assessment and interview, to screen for eligible 
and appropriate participants based on their levels of motivation and stability, among other 
factors. 


21 New York City Center for Economic Opportunity (2008). 

22 For more information about STED and its partner project ETJD, see Bloom (2015); Glosser, Barden, 
and Williams (2016); and Redcross et al. (2016). 

~ 3 In a joint decision, CEO, DYCD, and the research team excluded YAIP providers that also operated 
programs very similar to YAIP because of the high likelihood that young adults assigned to the control group 
would access those services. In addition, the team excluded a small number of providers because CEO and 
DY CD determined that those providers would be unable to comply with the study requirements and run the 
YAIP program effectively during the study period. One provider, Opportunities for a Better Tomorrow, served 
two YAIP cohorts in each cycle from different locations. Thus, 12 agencies operated 13 programs in the STED 
evaluation of YAIP. 

24 YAIP provider staff obtained parental consent for minors participating in the evaluation. 
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The YAIP evaluation enrolled a total of 2,678 young people in three consecutive co- 
horts, beginning with the July 2013 cohort, followed by the November 2013 cohort, and 
concluding with the March 2014 cohort. The research team randomly assigned 60 percent of the 
sample to the program group and 40 percent to the control group: 25 

• The program group. The 1,638 individuals who were randomly assigned to 
this group were offered YAIP program services, including a paid internship, 
job-readiness training, case management, and follow-up services. 

• The control group. The 1,040 individuals who were randomly assigned to this 
group were not offered YAIP program services, but were able to access other 
services that were available in the community, including other non-YAIP ser- 
vices offered at YAIP provider agencies. 

By measuring outcomes for both the program and control groups over time, it is possi- 
ble to assess whether YAIP services led to better outcomes for the program group than what 
would have happened in the absence of the program, as represented by the control group. Any 
statistically significant differences that emerge between the two groups will be considered 
“impacts,” or effects, of YAIP because, owing to the random assignment design, the research 
groups were comparable on both measured and umneasured characteristics at the time of study 
enrollment. 

The YAIP evaluation has three components: an implementation study, an impact study, 
and a benefit-cost study. The implementation study set out to describe the YAIP program, 
including the YAIP model as it was designed, the participants, the services that were ultimately 
offered to participants, the “dosage” of services that they received, and the local context and 
service environment in which the program operated. The implementation study relied on the 
following data sources: 

• Baseline data. YAIP staff collected background data on all sample members 
at the time of study enrollment. These data include information about age, 
gender, race and ethnicity, educational attainment, employment background, 
parenting status, living situation, receipt of public assistance, arrest history, 
and barriers to employment. 

An order to ensure that all open program slots were filled during the evaluation, the research team occa- 
sionally shifted the random assignment ratio to allow for a higher proportion of applicants to be assigned to the 
program group. For this reason, the final proportion of youth assigned to the program group is slightly above 
60 percent. To adjust for small variations in random assignment ratios and sample size between providers and 
cohorts, the team applied proportional weights to the impact analyses. These weights ensure that estimated 
impacts are not biased by differences across cohorts or providers in the proportion assigned to the program 
group. 



• Interviews and worksite observations. The research team visited each 
YAIP provider to conduct interviews with key program staff, including pro- 
gram coordinators, outreach workers, case managers, and job developers. 26 
Interviews were also conducted with YAIP management staff from CEO and 
DYCD. Additionally, each provider arranged for the research team to visit a 
worksite that was hosting YAIP interns. During these visits, the team inter- 
viewed employers and observed the worksite environment and operations. 

• Program participation data. DYCD provided data from its management in- 
formation system on young people’s participation in various YAIP program 
components, including job-readiness training, internship placements, case 
management services, and follow-up services. These data also include infor- 
mation about the employers that hosted YAIP interns, participants’ subsi- 
dized earnings, and participants’ outcomes at the end of their internships and 
after a nine-month follow-up period. 

• Participant questionnaire data. A member of the research team adminis- 
tered a short questionnaire to YAIP participants in two of the three study co- 
horts, visiting each provider during weekly educational workshops after the 
young people had been in their internship placements for at least one month. 
These questionnaires captured information about participants’ experiences at 
their internship placements and their level of satisfaction with those experi- 
ences. The research team collected a total of 570 completed questionnaires 
across all providers for an estimated response rate of 60 to 65 percent. 

• Employer or worksite supervisor questionnaire data. In an effort to leam 
more about the views of employers and worksite supervisors who took on 
YAIP interns, the research team at MDRC administered an online question- 
naire to all employers and worksite supervisors based on a list provided by 
DYCD. This questionnaire covered topics such as reasons for participating in 
YAIP, how YAIP participants compared with other entry-level workers, 
quality of support from YAIP providers, and the basis for hiring and termina- 
tion decisions. The team obtained a total of 145 responses for an estimated 
response rate of 15 to 20 percent. Given the low response rate, results from 
this questionnaire cannot be generalized to all internship employers or 
worksite supervisors and should be interpreted with caution. However, this 


26 0ne provider. Opportunities for a Better Tomorrow, operated the YAIP program at two different loca- 
tions in Brooklyn. The research team visited the Bushwick location, but did not visit the Bedford-Stuyvesant 
location. 
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report does touch on some findings from the questionnaire to help give in- 
sight into the employer or worksite supervisor perspective on the program. 

• In-depth interviews with YAIP participants. A member of the research 
team recruited eight YAIP participants from the second cohort to participate 
in in-depth interviews. The purpose of the interviews was to gain a deeper 
understanding of the obstacles disconnected young people face in engaging 
with school, work, or both; their reasons for participating in YAIP; their ex- 
periences with the program; and their progress after exiting the program. Af- 
ter a brief presentation about the in-depth interviews during orientation activ- 
ities at three different provider agencies, the team member randomly selected 
eight participants among those who volunteered. The research team selected 
the three providers from which participants were recruited based on their di- 
versity in geographical location, experience operating YAIP, and organiza- 
tional context. The research team conducted the interviews in person at three 
different points in time: once near the beginning of the program, once while 
internships were in full swing, and once a few months after internship place- 
ments for the cohort ended. 

As mentioned earlier, the evaluation’s impact study allows the research team to assess 
whether YAIP improved outcomes for disconnected youth. To this end, the research team 
estimated the program’s early impacts on key outcomes by measuring them approximately one 
year after participants enrolled in the study (unless otherwise specified). Selected based on a 
review of YAIP background materials and disconnected youth literature, these key outcomes 
fall into five domains: employment and earnings, education and training, psychosocial out- 
comes, economic and personal well-being, and criminal involvement. A later report will include 
longer-term impacts measured at approximately 30 months following study enrollment. The 
impact study relies on the following data sources: 

• Employment and earnings records. The research team used data from the 
National Directory of New Hires (NDNH) to measure quarterly earnings. 
Maintained by the federal Office of Child Support Enforcement, the NDNH 
contains data collected by state workforce agencies for jobs covered by un- 
employment insurance. These jobs include most formal employment, with 
the main exception of independent contract employment. Gaps in unem- 
ployment insurance coverage for independent contract employment are esti- 
mated to be 13 percent or higher. 27 


27 Hotz and Scholz (2002). 
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• Survey data. The survey firm Decision In formation Resources fielded two 
different survey instruments to sample members at two different points in 
time relative to young people’s enrollment into the study. These surveys are 
referred to as the 4-month survey and the 12-month survey, in accordance 
with when the fielding period for each began. 

Due to resource limitations, the 4-month survey was fielded only to the sec- 
ond study cohort and was completed by 719 of the 900 sample members in 
that cohort (443 program group members; 276 control group members), re- 
sulting in a response rate of 79.9 percent (80.4 percent of the program group; 

79.1 percent of the control group). 28 The 4-month survey contained questions 
about employment, education, and training; mental health and self-esteem; 
sense of control over one’s life; life challenges; social support and role mod- 
els; and social networks. Exploratory in nature, the 4-month survey was in- 
tended to capture information about the effects of working on young people’s 
well-being during the time period when most program group members were 
in their internship placements. 

The 12-month survey was fielded to all three study cohorts and was complet- 
ed by 2,127 of the full study sample of 2,678 (1,327 program group mem- 
bers; 800 control group members), resulting in a response rate of 79.4 per- 
cent (81.0 percent for the program group; 76.9 percent of the control group). 

The 12-month survey contained questions about service receipt; employ- 
ment, education, and training; household composition, income, and material 
hardship; health, well-being, and psychosocial outcomes; criminal history; 
and social support and networks. (See Appendix A for more information 
about the survey samples and an analysis of the extent to which results may 
be biased by survey nonresponse for either survey.) 

• Postsecondary school enrollment data. The research team collected data 
from the National Student Clearinghouse database, which includes infor- 
mation on enrollments in most two- and four-year postsecondary institutions 
in the United States. 

The third component of the evaluation, the benefit-cost study, will assess how YAIP’s 
costs compare to its benefits. Findings from the benefit-cost study will be presented in a later 
report. 


28 The research team chose the second cohort for the sake of convenience rather than any particular belief 
in a systematic difference between cohorts. 
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Roadmap to the Report 

The remainder of this report is divided into the following chapters. Chapter 2 describes YAIP’s 
background, sample recruitment and screening practices, and characteristics of the study sample 
at the time of study enrollment. Chapter 3 explains the program model and assesses its imple- 
mentation. Chapter 4 presents differences in services received by program group members 
compared with control group members, as well as early impact findings in five key domains: 
employment and earnings, education and training, psychosocial outcomes, economic and 
personal well-being, and criminal involvement. Chapter 5 concludes the report and discusses 
the next steps in the YAIP evaluation. 
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Chapter 2 


Background, Recruitment, Intake, and Sample 

Characteristics 


The New York City Department of Youth and Community Development (DYCD) developed 
the Young Adult Internship Program (YAIP) in order to provide workforce support services to 
an underserved subset of the city’s more job-ready disconnected youth. This chapter describes 
YAIP’s origins, structure, and recruitment and intake processes, as well as the background 
characteristics of the study sample. 


Background 

Nearly 180,000 New York City residents between the ages of 16 and 24 are neither working nor 
in school. 1 The proportion of disconnected youth in the five boroughs of New Y ork City, at 1 7.2 
percent, exceeds the national average of 12.3 percent. 2 Further, the New York City metropolitan 
area fares poorly on this measure relative to other large American metropolitan areas, including 
Boston, the District of Columbia, Chicago, Baltimore, and Los Angeles. 3 

Disconnection does not affect youth across New York City equally. In certain areas of 
the Bronx, for example, about a third of young people are disconnected from school and work, 
in stark contrast with areas of Manhattan, where disconnection characterizes only 3.2 percent of 
young adults. 4 In line with national trends, large disparities also appear along racial and ethnic 
lines: In the New York City metropolitan area, the rates of disconnection among Black and 
Latino young people are 18.2 and 15.6, respectively, compared with a rate of 8.7 percent among 
whites. 5 


As discussed in Chapter 1 , disconnected youth are particularly vulnerable to long-tenn 
economic hardship. In an effort to improve the future prospects of this population, DYCD 
developed YAIP. The origins of YAIP date back to 2005, when a team of DYCD program 
directors convened to develop a program that would target the many young people who they 
believed were underserved by the agency’s two major employment programs at the time, the 


'Calculations based on data from Measure of America of the Social Science Research Council (2017). 
Calculations based on data from Measure of America of the Social Science Research Council (2017); 
Burd-Sharps and Lewis (2017). 

3 Burd-Sharps and Lewis (2017). 

4 Data2go.nyc (2014). 

5 Burd-Sharps and Lewis (2017). 
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Summer Youth Employment Program (SYEP) and the Out-of-School Youth (OSY) Program. 
SYEP provides young people between the ages of 14 and 24 with paid summer employment for 
up to six weeks in July and August, while OSY, highlighted in Chapter 1, is a yearlong occupa- 
tional skills training program for low-income disconnected youth between the ages of 16 and 
24. The DYCD program directors reasoned that a substantial subset of discomiected young 
adults could benefit from an intervention that provided more support than SYEP, but fewer 
services than OSY. 

Borrowing and blending elements from both SYEP and OSY, the DYCD team con- 
ceived of a paid internship program that targets the most job-ready individuals among the city’s 
disconnected young adult population. The target population for YAIP is discomiected young 
people who are not so disadvantaged that they need intensive wraparound services, but disad- 
vantaged enough that they require the help of a program to reengage in school or work in order 
to improve their future prospects. 6 Around the same time that DYCD developed YAIP, the New 
York City Center for Economic Opportunity (CEO) — charged with the implementation, 
monitoring, and evaluation of the city’s anti-poverty agenda — was looking to expand work- 
force engagement opportunities for disconnected youth. CEO agreed to allocate $7 million to 
DYCD to make YAIP a reality, and since YAIP’s inception, has worked closely with DYCD to 
administer the program. YAIP was rolled out in 2007 at 15 different community-based organi- 
zations selected through a competition and served 900 young adults across the city in its first 
year of operation. Today, the program operates out of 14 different provider agencies and serves 
about 1,800 young people each year. 7 


Program Structure 

YAIP is a multiphase program that runs three overlapping program cycles each year. The 
program enrolls participants in cohorts, and thus members of a particular cohort move through 
the program together. Each YAIP provider is responsible for enrolling and serving a portion of 
the full cohort, usually about 30 young adults per location. In bringing together a group of 
disconnected young people looking to make a positive change in their lives, YAIP aims to 
create a new, prosocial peer network. This social network is also meant to serve as a support 
group whereby participants can share experiences in their internships and work through chal- 
lenges together. Moreover, the cohort structure encourages a sense of accountability to the 
group among participants, resulting in greater investment in each other and in the program. For 


6 While usage of the term “wraparound services” may vary slightly, it is generally meant to indicate an 
intensive, individualized, holistic approach to service provision. 

7 DYCD was able to expand the number of program slots with additional funding from the New York City 
Young Men’s Initiative. 
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YAIP staff, the cohort model allows them to get to know participants well because they work 
intensively with them as a group that progresses through the program concurrently. 

Figure 2.1 depicts the timing and flow of YAIP’s overlapping program cycles during 
the study period. The three phases of each program cycle are summarized below: 

• Phase 1: Orientation. The first two to four weeks of the program (the exact 
duration varies by provider) are referred to as the orientation phase, when 
participants are expected to attend daily workshops facilitated by program 
staff at provider offices. Participants are paid minimum wage for 25 hours 
per week during this phase, and workshops are typically five hours per day. 8 
The goals of orientation are to prepare participants for the workplace by 
providing various job-readiness and personal development activities, to lay a 
foundation for cohort cohesion using icebreakers and group activities, and to 
match participants’ interests and skills with an available and appropriate in- 
ternship. 

• Phase 2: Internship. During the 10 to 12 weeks of this phase, youth are ex- 
pected to work 20 hours a week in their internship and continue to earn min- 
imum wage. DYCD fully subsidizes their earnings. Internship placements are 
at a wide variety of worksites, running the gamut from the offices of local 
politicians to commercial drugstores, social service nonprofit agencies, cloth- 
ing retail chains, and a local radio station. Regardless of worksite, most 
young people are assigned tasks that involve clerical work, customer service, 
or maintenance. The goals of the internship phase vary based on the particu- 
lar needs of young people, but generally include work experience, develop- 
ment of soft or hard skills, career exploration, and potential transition from a 
subsidized internship to a permanent, unsubsidized position. Once a week, 
youth are required to return to the provider offices to attend a five-hour edu- 
cational workshop, for which they are also paid minimum wage. These 
workshops cover topics including job readiness, healthy living, money and 


8 The first cohort in the evaluation completed their internships in 2013, when the minimum wage in New 
York State was $7.25 per hour. The second cohort’s participation straddled 2013 and 2014, when the minimum 
wage was raised to $8.00 per hour. The third cohort completed their internships in 2014 and earned the 
increased minimum wage of $8.00 throughout their participation in the program. As of the publication of this 
report, the minimum wage in New York City is $11 per hour for employers of 11 or more employees and 
$10.50 for employers of 10 or fewer employees. 
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Figure 2.1 
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time management, and conflict resolution. Weekly workshops also help nur- 
ture the cohort bonds initiated during Phase 1 and provide youth with an op- 
portunity to speak with their case managers if they need support or guidance. 

• Phase 3: Follow-Up. This phase covers the nine months following partici- 
pants’ completion of their internship. During this time, providers are ex- 
pected to help participants secure and maintain an “outcome placement.” 
Outcome placements include participation in unsubsidized employment, ed- 
ucation, training, or the military. Staff follow-up efforts largely focus on try- 
ing to maintain contact with participants in order to share job leads, help 
connect them to needed services, and provide other case management and 
counseling support. 


Eligibility 

YAIP participants must be New York City residents, 16 to 24 years old, eligible to work in the 
United States, and neither in school nor working. In an effort to pinpoint the most job-ready 
disconnected young people, additional eligibility criteria were devised to screen out severely 
disadvantaged young adults, as well as young adults that do not need program services to 
improve their employability. Participants must score at least a sixth-grade reading level on the 
Test for Adult Basic Education (TABE), which is administered as part of the application 
process. They cannot have a postsecondary degree and must observe a semester-long waiting 
period if they recently graduated from high school or dropped out of high school or college. (In 
theory, this waiting period gives these young adults time to reengage in school or work on their 
own.) Additionally, young people cannot have participated in a CEO-funded paid internship 
program at any point in the past. 9 

• Among applicants who meet YAIP’s formal eligibility criteria, provider 
staff reported favoring those who demonstrate higher levels of stability 
and motivation. At the same time, staff seek out young people who face 
significant disadvantages. 

Beyond the formal eligibility criteria, staff reported targeting certain applicant charac- 
teristics to best recruit the more job-ready subset of the discomiected youth population. Ideal 
applicants, according to most program staff, are motivated and eager to engage with what the 
program has to offer. Staff described these youth as “future-oriented,” “receptive,” and “looking 
for a change in their life.” Staff also place a premium on candidates who demonstrate a certain 

9 Specifically, young people were ineligible for YAIP if they previously participated in Justice Communi- 
ty, NYC Justice Corps, Project Rise, or the Work Progress Program. 
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threshold of stability, particularly in the areas of housing and child care, which allows them to 
participate fully in the program. 

For example, one provider observed that the most successful participants tend to be 
those who live at home with a parent who is mentally stable and who is working, very support- 
ive, or both. Another provider shared an anecdote about an applicant who was a great fit for 
YAIP, except that the applicant did not have reliable child care for her three children. Thus, the 
provider deferred her application. On the other hand, staff are disposed toward young adults 
who have significant disadvantages. Put differently, they seek out young adults who stand to 
benefit the most from YAIP. Among this group are first-generation high school graduates and 
people with limited work experience. Program staff assess these eligibility preferences, as well 
as the formal eligibility criteria, through an extensive intake process, described later in this 
chapter. 


Recruitment 

Recruitment for YAIP is ongoing — YAIP and provider agency staff are always on the lookout 
for future applicants, and they field referrals from individuals familiar with the program year- 
round. However, the full-force, active recruitment phase reaches its height in the two weeks 
between program cycles, when staff have more time to focus their efforts. By design, providers 
collect more completed applications than they have program slots. In this way, providers create 
a larger pool of applicants from which to choose and a buffer against the inevitable applicant 
attrition leading up to the program start date and continuing into the orientation phase. Ultimate- 
ly, each provider enrolls enough applicants to fill a prescribed number of program slots, ranging 
from about 30 to 45. 

Each YAIP provider recruits applicants independently, but they all use similar strate- 
gies. Staff post on bulletin boards and internet job search sites, distribute flyers, present on the 
program at community events, and spread the word via social media. The paid internship is a 
key selling point in marketing the program to potential participants. Beyond the prospect of 
earning a wage, applicants are attracted by the particular kinds of internships that the program 
offers. For example, an internship with a local radio station draws a great deal of interest. 
Providers also rely heavily on referrals and word of mouth. Indeed, staff reported that their best 
applicants are those referred to YAIP by past participants, worksite partners, or organizations 
such as churches or other nonprofits that are already familiar with the program. This familiarity 
with YAIP also likely results in some pre-screening for applicants’ suitability. 

YAIP providers are required to enroll at least 80 percent of participants from targeted 
high-need community districts, characterized by high poverty rates. Figure 2.2 presents a map 
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Figure 2.2 
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of New York City that delineates the high-need community districts. The locations of YAIP 
provider agencies participating in the evaluation are also marked on the map. During the study 
enrollment period, DYCD and CEO agreed to relax their expectations around recruiting from 
high-need community districts. Providers welcomed this change, as many staff members 
thought that the requirement made recruitment more difficult by limiting the pool of potential 
applicants to a certain geographic area (as opposed to the entire city) and that it did not neces- 
sarily yield candidates with greater needs. 

Providers recruit most heavily from within the neighborhood or community districts 
they serve, but they receive applications from young adults all across the city. One provider 
located in the relatively isolated Washington Heights neighborhood of Manhattan, for instance, 
served young people who commuted from Brooklyn to participate in the program. According to 
staff, some participants may not be able to find the services they need in their own communities, 
or they intentionally look for opportunities outside of their neighborhoods. Interestingly, staff 
reported that participants who come from further away are highly motivated to be a part of 
YAIP and often wind up being the most engaged participants. 

• YAIP’s eligibility criteria and lengthy recruitment and intake processes 
require a high level of effort from provider staff in order to successfully 
enroll young people who are appropriate for the program in sufficient 
numbers. YAIP’s overlapping program cycles pose an additional chal- 
lenge. 

YAIP providers face a number of recruitment challenges. First, the recruitment and in- 
take process is complex and lengthy. Thus, providers find it challenging to time recruitment 
such that applicants will not be accepted so far in advance of orientation that they lose interest in 
or find an alternative to YAIP, while also not beginning recruitment so late that provider staff 
must struggle to meet recruitment goals. The random assigmnent process compounded this 
challenge during the study period, increasing both the complexity and the duration of the 
recruitment process, as well as increasing recruitment target numbers. Further, YAIP’s eligibil- 
ity requirements narrowed what appeared, at first glance, to be a large potential applicant pool 
of disconnected youth: As described earlier, applicants must meet formal eligibility criteria and 
be deemed good matches for YAIP based on more subjective criteria. Finally, though staffing 
practices vary slightly across providers, the YAIP program at each location employs a small 
staff of around five people. YAIP’s overlapping program cycles result in staff members simul- 
taneously operating the program and recruiting for the next cycle, leaving them stretched thin 
and unable to dedicate as much time to recruitment as they would like. Chapter 3 describes 
YAIP staffing in more detail. 
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Intake 

All YAIP applicants are required to complete a multistage intake process, which is designed to 
screen for eligible and appropriate participants. The intake process comprises three main 
components. First, applicants complete the TABE and a program application, which covers 
basic demographic infomiation, eligibility, and required documentation. Second, applicants 
complete the self-assessment form, which asks about their employment and educational history, 
interests, and service needs. Finally, applicants undergo an intake interview, usually conducted 
by a case manager or program coordinator. Typically, applicants visit the program offices at 
least three times to complete the three intake components. As a result, the intake process spans 
several days and tests applicants’ motivation to enroll in the program. 

YAIP providers tailor intake procedures in varied and creative ways to help assess pro- 
gram candidates’ readiness to engage in program services. Some of the providers hold basic 
job-readiness workshops in the weeks leading up to enrollment. LaGuardia Community 
College, for example, runs a few job-readiness workshops before program orientation in which 
they instruct young people in cover letter writing and administer a career interest survey. These 
sessions serve the dual purpose of assessing young adults’ level of commitment to the program 
and ensuring all applicants receive at least some help, even if they are not accepted into the 
program. Other providers, such as BronxWorks, give applicants a checklist of activities, such as 
obtaining a library card and finding job leads, to complete before enrollment. In line with the 
cohort model, several providers organize group activities or group interviews to observe how 
well applicants interact with their peers. In addition to helping staff assess applicants’ attitudes 
and motivations, these activities accelerate familiarity between program staff and the young 
people, which makes it easier for them to make the transition into the workshop curriculum and 
internship matching process during the first phase of the program. 

After completing the intake activities and just before the start of orientation, applicants 
officially enroll in YAIP. However, their continued enrollment in the program is contingent 
upon their demonstrated ability to successfully complete the internship, which is assessed 
during orientation. For example, program staff may decide to de-enroll an individual for reasons 
such as poor attendance or because the participant’s circumstances change, making it impossi- 
ble for them to fully engage in the program. 10 Rather than allow program slots to go unfilled, 
staff replace de-enrolled participants with other eligible applicants from a waiting list. This 
waiting list is the result of providers’ practice of recruiting a larger pool of applicants than there 
are program slots as a cushion against attrition, as described earlier in this chapter. 


10 De-enrolled applicants are not counted in the providers’ performance measures, but they remain part of 
the program group for the purposes of the evaluation. 
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The Effects of the Evaluation on Recruitment and Enrollment 

Figure 2.3 illustrates the recruitment, intake, and random assignment processes as they occurred 
during the evaluation. The evaluation employs a random assigmnent research design, which 
places an inevitable burden on programs and their staff. Most notably, YAIP providers were 
required to recruit nearly twice as many applicants given that only 60 percent of applicants were 
assigned to the program group. Additionally, providers had to modify their recruitment, intake, 
and enrollment procedures in order to accommodate the additional applicants required for the 
study. Conscious of the extra demands that the evaluation placed on providers, CEO awarded 
each participating YAIP provider up to $37,000 in additional funds during the study’s enroll- 
ment period. Most of the providers spent the money on additional staff to support the expanded 
recruitment and application process. DYCD also furnished providers with extra TABE materi- 
als and MetroCards (for use on public transportation) to help them accommodate the larger 
number of applicants. MDRC offered technical assistance regarding recruitment strategies, 
which some of the providers accepted. 

Providers expanded their traditional catchment areas in an effort to meet their increased 
target numbers. Most of the YAIP providers were unable to fully implement the extensive 
screening procedures they typically use to select potential participants. For example, one 
provider did away with its pre-enrollment checklist of activities, and another provider cut its 
intake process from five steps to three. Moreover, staff were no longer able to handpick their 
participants to curate the overall intragroup dynamic, which many staff regard as an important 
aspect of the program model. 

YAIP provider staff, DYCD, and the research team collaborated to schedule random as- 
sigmnent to take place for each cohort shortly before the program start date to give providers 
enough time to implement their usual screening procedures while ensuring that participants 
would begin orientation as soon as possible once they were randomly assigned to the program 
group. The intent was to minimize attrition by screening out less motivated applicants before 
random assignment and program enrollment and by promptly engaging program group mem- 
bers in program services. In reality, providers experienced greater attrition before (and, to a 
much lesser extent, during) the orientation period than anticipated. Indeed, most of the providers 
invited all of the applicants on their waiting list to attend orientation, which in turn required one 
or sometimes two more rounds of random assigmnent (as shown in Figure 2.3) to back-fill 
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Figure 2.3 
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program slots. 11 As a result, 13 percent of the sample was enrolled into the study after program 
orientation began, meaning that some program group members missed out on some early 
program services. 

Despite changes to the recruitment and enrollment process to accommodate the study, 
evidence suggests that the cohorts involved in the study did not differ meaningfully from other 
YAIP cohorts. An analysis of outcome placement data for all YAIP cohorts served during the 
program’s first eight years of operation found that the outcomes for the cohorts participating in 
the study did not differ from those for other YAIP cohorts. 


Sample Characteristics at the Time of Study Enrollment 

This section describes the demographics and background characteristics of the YAIP study 
sample and compares the sample members with the larger population of disconnected youth in 
New York City and the United States, as well as Project Rise sample members, in order to 
situate the YAIP sample within the overall context of youth disconnection. Project Rise, another 
CEO program for disconnected youth, is described in Chapter 1. Table 2.1 presents selected 
baseline characteristics of the YAIP study sample, reported by youth at the time they enrolled 
into the study. 12 

Reflective of the gender makeup of the overall population of disconnected youth in 
New York City, the sample is divided about evenly between men and women. 13 The vast 
majority of sample members are Black, non-Hispanic (58 percent), or Hispanic (36 percent). 
Over half of the sample is 19 to 21 years old, which is consistent with reports from provider 
staff that the middle of the eligibility age range of 16 to 24 is most suitable for YAIP and the 
age group that they prefer to serve. 


"During the study enrollment period, providers’ waitlists were composed of applicants who had already 
gone through random assignment and been assigned to the program group. When waitlists were exhausted, 
additional rounds of random assignment were conducted for other eligible applicants. Those applicants 
assigned to the program group back-fdled remaining empty program slots. 

12 Young people reported baseline data to YAIP providers as part of their standard intake process and pro- 
viders subsequently shared these data with the research team. Therefore, provider staff did not receive any 
special training or guidance from the research team about recording these data, nor was data quality regularly 
monitored. As a result, certain measures may not be entirely reliable. In particular, educational attainment 
measures showed some inconsistencies. 

13 Treschan and Molnar (2008). 
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Table 2.1 


Characteristics of Sample Members at Baseline 


Characteristic 

Total 

Average age (years) 

20.7 

Age (%) 

Under 19 years old 

14.3 

19-21 years old 

51.5 

22-24 years old 

34.1 

Male (%) 

49.1 


Race (%) 

Hispanic 

Black, non-Hispanic 
Other 

Highest degree achieved 3 (%) 


No degree 38.3 

GED certificate 7.3 

High school diploma 32.7 

Some postsecondary education, but no postsecondary degree 21.3 

Bachelor's degree or higher 0.4 

Does not have high school diploma or GED certificate (%) 38.3 

Among those without high school diploma or GED certificate 
Ever enrolled in GED program (%) 39.6 

Among those ever in GED program, years since last enrolled 1.5 

Years since last in school 2.4 

Ever employed (%) 71.7 

Among those ever employed, time since last employed (%) 

Six months or less 35.9 

Six months to less than 12 months 26.9 

One year to less than 2 years 20.4 

Two years or more 16.8 

Has not worked in a job for three or more months (%) 69.9 

Median months since last in school, enrolled in GED program, or working 9.1 

Has a child (%) 20.0 

Among those with a child, number of children 11 1 .3 


(continued) 


36.2 

58.1 

5.7 
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Table 2.1 (continued) 


Characteristic Total 


Living situation (%) 

Living with parents 59.9 

Staying with someone else, such as friends or relatives 25.0 

Rents or owns home 6.2 

Supervised living 0 2.1 

Homeless 2.0 

Other living arrangement 4.7 

Receives public assistance (%) 26.3 

Food stamps (SNAP) 23.2 

Family assistance (TANF) 3.4 

Disability (SSI) 1.9 

Other income 0 * 1.7 

Ever arrested (%) 26.0 

Has any of the following barriers to employment (%) 42.3 

Limited literacy or math skills 13.0 

Ever runaway, homeless, or living in temporary or emergency housing 5.9 

Pregnant or has child 21.6 

Has a mental or physical disability 4.3 

Ever convicted of a crime 8.2 

Sample size 2,678 


SOURCES: MDRC calculations based on data from MDRC's random assignment system and the 
YAIP management information system. 

NOTES: GED = General Educational Development. 

TANF = Temporary Assistance for Needy Families. 

SNAP = Supplmental Nutrition Assistance Program. 

SSI = Supplemental Security Income. 

“Students who obtained a high school certificate of completion but not a high school diploma or 
GED certificate are shown as having no degree. 

b The largest entry permitted for number of children was "three or more." For the purpose of 
calculating the mean number of children among parents in the study sample, those who fell in this 
category were counted as having three children. Thus, the figure presented may slightly understate the 
actual mean. 

“Supervised housing refers to a range of living situations that are closely monitored by a public or 
private agency. Examples include supervised independent living, emergency housing, work-release 
facilities, and halfway houses. 

d Other public assistance income sources include safety net assistance, unemployment 
compensation, and other unspecified sources of income. 
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• Young people in the study sample compare favorably to the overall 
population of disconnected youth in New York City on various socioeco- 
nomic indicators, suggesting that providers successfully targeted the 
city’s more job-ready subset of disconnected youth. 

Sixty-two percent of the YAIP study sample had earned at least a high school diploma 
or equivalency certificate, compared with only half of the overall population of disconnected 
youth in New York City. 14 This somewhat higher percentage is consistent with the program’s 
goal of serving the most job-ready subset of the disconnected youth population. Similarly, a 
sizable portion of the sample (21 percent) had some college experience. Among the 38 percent 
of young people in the sample without a high school diploma or equivalency certificate, the 
average length of time since they were last in school was 2.4 years. Forty percent of them had 
enrolled in a GED program at some point before joining YAIP. 

Nearly three-fourths of the sample had previous work experience (compared with 65 
percent of the Project Rise study sample), 15 but about two-thirds of this group had not been 
employed in over six months and nearly all had not worked in any job for three months or 
longer. The median length of disconnection — that is, not in school, not enrolled in a high 
school equivalency program, and not employed — among young people in the study sample 
was nine months. 

As mentioned previously, YAIP staff consider applicants with a certain degree of stabil- 
ity in their lives, particularly in the areas of housing and child care, to be most appropriate for 
the program. Even though housing is a significant challenge among the broader population of 
disconnected youth — and perhaps particularly so in New York City, with its unforgiving 
housing market — only about 4 percent of the YAIP sample reported experiencing unstable 
housing at intake, reflected in the percentages in a supervised living situation or homeless. Most 
sample members lived with their parents (60 percent), while one-fourth reported staying with 
friends or other relatives. Similarly, the proportion of the sample that has children (20 percent) 
is smaller than in other studies of disconnected youth (figures range from 28 to 34 percent). 16 

Over one-fourth of the sample received some form of public assistance, most common- 
ly from the Supplemental Nutrition Assistance Program (SNAP, or formerly food stamps). As 
noted above, the majority of the sample was living with their parents or in someone else’s home 
when they enrolled in the study. For these participants, it is unclear whether reports of public 
assistance receipt were based on the household in which they were living or on their own 


14 Treschan and Molnar (2008). 

15 Manno, Yang, and Bangser (2015). 

16 Manno, Yang, and Bangser (2015); Burds-Sharp and Lewis (2017). 
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personal receipt of public assistance. Moreover, those who answered based on their household 
may not have been fully aware of the receipt of public assistance. These caveats aside, the rate 
of public assistance receipt among members of the YAIP study sample is relatively low com- 
pared with that among members of the Project Rise study sample. Less than 4 percent of the 
YAIP sample was receiving Temporary Assistance for Needy Families at the time of enroll- 
ment, compared with 15 percent of the Project Rise sample. SNAP receipt was also much more 
prevalent among the Project Rise sample (58 percent) compared with the YAIP sample (23 
percent). 17 These sample differences are expected given YAIP’s goal of serving a less disadvan- 
taged group of young people than those served by Project Rise. 

Beyond their limited education and employment history, over 40 percent of sample 
members faced at least one other significant barrier to employment, such as pregnancy or 
parenting responsibilities (22 percent), limited literacy or math skills (13 percent), or a mental or 
physical disability (4 percent). Approximately one in four sample members had been arrested 
and 8 percent had been convicted of a crime. Again, these figures suggest that YAIP sample 
members, on average, encounter fewer barriers to employment than other populations of 
disconnected youth. For example, in the Project Rise study, almost half of the sample had a 
history of arrest, 18 and in a nationwide sample of disconnected youth, 15 percent had a disabil- 
ity. 19 


Conclusion 

CEO and DYCD created YAIP to fill a gap in services for the most job-ready disconnected 
youth in New York City. The program’s designers carefully and intentionally crafted the 
program’s eligibility criteria to target young adults who need a limited amount of programmatic 
support to reconnect with school or work. YAIP providers understand this goal and strive to be 
equally careful and intentional in selecting participants they believe stand to benefit most from 
the program. 

Recruiting and enrolling young adults who are eligible and appropriate for YAIP pose 
significant challenges for provider staff, in part because of the program’s long application 
process, and in part because of the program’s overlapping cycles. The short period (about two 
weeks) between the conclusion of one cohort’s internship period and the start of orientation for 
the next cohort requires YAIP staff to take an “all hands on deck” approach to the lengthy and 
involved application process. They must also recruit continuously because applicants have 


17 Manno, Yang, and Bangser (2015). 
18 Manno, Yang, and Bangser (2015). 
19 Burds-Sharp and Lewis (2017). 
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competing priorities and interests that prevent them from committing to the program. In the 
final days before orientation, many providers must scramble to determine whether they have the 
requisite number of finalized applications to fill their program slots. 

The study somewhat altered the providers’ screening procedures. Most providers had to 
condense their process in order to obtain the minimum number of candidates, and random 
assignment meant that staff lost their ability to curate a cohort’s composition. Nevertheless, the 
characteristics of the study sample suggest that YAIP continued to serve its intended target 
population. Compared with the local and national populations of disconnected youth, as well as 
Project Rise sample members, YAIP sample members were more advantageously positioned on 
various socioeconomic indicators. They were more likely to have a high school credential and 
previous work experience and less likely to be raising children of their own, and very few were 
in unstable housing situations at the time of their enrollment into the study. Nevertheless, YAIP 
study sample members still faced significant challenges. 
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Chapter 3 

The YAIP Model and Its Implementation 


The Young Adult Internship Program (YAIP) provides disconnected young adults in New York 
City between the ages of 16 and 24 with a paid internship, educational workshops, and case 
management services. The goal of the program is to provide job-ready disconnected youth with 
supports that will lead to long-term reengagement in productive activities, such as employment, 
school, and training. Based on interviews with provider staff, management staff, and partici- 
pants; questionnaires administered to provider staff, participants, and worksite staff; participa- 
tion data from DYCD’s management information system; and worksite observations by the 
research team, this chapter describes how provider staff implemented the program during the 
YAIP study period. 


Program Structure and Staffing 

The evaluation includes 12 YAIP providers that delivered the program in 13 locations across 
the Bronx, Brooklyn, Manhattan, and Queens. 1 The providers in the evaluation bring a diverse 
set of strengths to the program, stemming, at least in part, from the type of agency that houses 
YAIP and the way it staffs the program. Table 3.1 offers a basic description of these 12 provid- 
ers. 2 Several of the provider agencies focus solely on serving young adults (for example, 
Opportunities for a Better Tomorrow and The Door), but more often YAIP providers are 
community-based organizations that offer a range of services to all members of the community, 
with YAIP housed in their workforce development divisions. 

• DYCD gives providers leeway in how they configure staffing for YAIP; 
as a result, each provider staffs the program somewhat differently. 

Each provider agency is required to fulfill the following six capacities for YAIP: re- 
cruitment, worksite development, case management, retention services, educational services, 
and data entry. Providers vary in how they distribute these responsibilities. For example, certain 
providers employ a dedicated job developer to manage worksite partnerships or a dedicated 


One YAIP provider. Opportunities for a Better Tomorrow, operates the program in two locations and will 
be considered one entity when agency-level differences are paramount (for example, in discussions about 
provider resources) and separate entities when location-based differences take precedence (for example, in 
discussions about participant demographics). 

2 Much of the information included in this table was self-reported by provider organizations on their re- 
spective websites. 
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Table 3.1 

YAIP Provider Information 


Provider Name 

Provider Description 

Services and Programs Offered 

Target Group 

Size 

BronxWorks, 

Inc. 

Multiservice, nonprofit organization 
that supports families and individu- 
als in the Bronx by aiming to 
improve their economic and social 
well-being 

Benefit assistance, children and youth 
programs, immigration services, 
homeless services, eviction prevention, 
services for seniors, aid for people with 
chronic illnesses, and workforce 
development 

Individuals and 
families of all ages 
with broad needs 

40,000 Bronx 
residents 
served in 2015 




Children, youth, 
adults, seniors, 

110,000 
individuals in 
Brooklyn and 
Queens served 
in 2013 

Catholic Chari- 
ties Neighbor- 
hood Services, 
Inc. 

Faith-based organization that seeks 
to provide social services to 
struggling New Yorkers 

Services for children, families, and older 
adults; behavioral health, developmental 
disability, and housing services 

individuals with 
developmental 
disabilities, individu- 
als with mental 
illness, and the 
isolated 


Cypress Hills 
Local Develop- 
ment Corpora- 
tion 

Nonprofit development organiza- 
tion helping the local residents of 
Cypress Hills, Brooklyn, achieve 
educational and economic success 

Career services, professional training, 
adult education classes, housing and 
homeownership services, family and 
youth recreational services, after-school 
programs, summer camps 

Residents of all ages, 
with many programs 
for youth 

9,000 

residents of 
Cypress Hills 
in 2015 



Crisis and mental health counseling, 



The Door 

Organization that provides compre- 
hensive youth development services 
in order to empower NY C youth to 
reach their full potential 

health services, legal assistance, high 
school equivalency (HSE) and English 
for Speakers of Other Languages 
(ESOL) classes, tutoring, career 
development services, recreational 
activities, nutritious meals 

Disconnected youth 

10,000 young 
people per 
year 
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Table 3.1 (continued) 


Provider Name 

Provider Description 

Services and Programs Offered 

Target Group 

Size 

Greater Ridge- 
wood Youth 
Council, Inc. 

Nonprofit organization that seeks to 
improve the quality of life for youth 
and their families in Queens by 
providing educational, recreational, 
cultural, and employment programs 

After-school programs, summer youth 
employment, adult literacy classes 

Neighborhood youth 
and their families 

6,000 children 
and families 
per year 

Henkels and 
McCoy, Inc. 

National construction company that 
offers training services 

Summer youth programs, technology 
and hands-on training 

School-aged children 
and high school-aged 
youth 

420 youth per 
year 

Mosholu Mon- 
tefiore Commu- 
nity Center, Inc. 

Community center that aspires to 
support, educate, and enrich the 
lives of community members 
through education and recreational 
programs and services in the Bronx 
and Manhattan 

Day care, camp, sports, teen programs, 
case assistance, College Bound program 
(free college trips, SAT preparation, 
college counseling) 

Preschoolers, school- 
aged children, teens, 
adults, and senior 
citizens 

35,000 Bronx 
and Manhattan 
residents per 
year 

Northern 

Manhattan 

Improvement 

Corporation 

Community-based nonprofit that 
aspires to stabilize the community 
and help residents build a better life 

Legal counseling, community organiz- 
ing, adult education and workforce 
development, health education, domestic 
violence intervention, benefit assistance 

Residents of the 

Upper Manhattan 
neighborhoods of 
Inwood and Wash- 
ington Heights 

12,000 
residents per 
year 

Opportunities for 
a Better Tomor- 
row 

Organization that seeks to help 
individuals achieve self-sufficiency 
and financial security 

Job training, academic reinforcement, 
HSE/General Educational Development 
(GED) and English as a Second 

Language (ESL) classes, job placement 
and support services 

Disadvantaged youth 
and adults 

Over 900 
youth and 

3,000 adults 
served in 2015 
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Table 3.1 (continued) 


Provider Name 

Provider Description 

Services and Programs Offered 

Target Group 

Size 

The Research 
Foundation at 
LaGuardia 
Community 
College 

College serves the community by 
providing education in a variety of 
fields 

Academic courses, HSE and ESL 
classes, various certificate and licensing 
programs, fatherhood program, and free 
job training 

Young adults and 
adults 

Approximate- 
ly 45,000 
enrolled in 

Fall 2015 

SCAN NY 

Nonprofit youth and family service 
provider in East Harlem and the 
South Bronx that aims to empower 
at-risk youth by focusing on 
inspiring kids to achieve positive 
goals 

Early childhood education, substance 
abuse treatment, violence prevention, 
mental health services, academic and 
college test preparation, after-school 
activities and events, employment skills 
training, and resume building. 

High-risk youth and 
their families 

7.000 children 
or teens and 

1.000 adults 
each year 

SCO Family of 
Services 

Community-based human services 
organization in New York City and 
Long Island that seeks to support 
personal well-being and give 
everyone the opportunity to succeed 

Early childhood education; home visits; 
mother-child residences; youth after- 
school, summer, and school day 
enrichment programs; counseling for 
LGBTQ and homeless youth, residential 
programs for individuals with special 
needs 

Homeless families, 
struggling teenagers, 
at-risk individuals, 
disabled adults, 
young women, 
mothers, and children 

60,000 New 
York children, 
youth, adults, 
and families 
each year 


SOURCES: Provider websites and MDRC interviews with provider staff. 

NOTE: The range and quality of resources provided to YAIP participants varied by provider and was not necessarily related to the 
size of the provider or the number of in-house services and programs they offered. 



workshop facilitator to run the educational workshops. More commonly, YAIP employees wear 
many different hats, with everyone playing some role in all six areas. On average, each YAIP 
provider employs about five full-time staff. As shown in Figure 3.1, the YAIP line staff at each 
provider agency report to a YAIP program coordinator. The YAIP program coordinator is a 
full-time, YAIP grant-funded employee who handles the site’s day-to-day management and 
implementation of YAIP. The program coordinator, in turn, reports to the provider agency’s 
director of workforce programs (or someone in a similar role). Each provider operates under the 
supervision of one of three DYCD program managers who monitor contract compliance and 
support the providers in achieving the program’s goals. The program managers report to the 
YAIP director, a DYCD employee who works closely with CEO to oversee the implementation, 
monitoring, and evaluation of the program. 

YAIP providers operate their programs independently of one another. However, DYCD 
encourages providers to view themselves as part of the larger YAIP program and brings them 
all together for bimonthly meetings. Nevertheless, collaboration among providers was minimal 
and comparisons among them can invite competition. Some program coordinators thrive on this 
competition; others wish for a greater exchange of knowledge and resources. 

• DYCD’s performance expectations, YAIP’s structure, and the challeng- 
es associated with serving disconnected youth leave staff vulnerable to 
burnout. Provider staff reported that frequent turnover poses a chal- 
lenge to the successful implementation of YAIP. 

DYCD established explicit performance expectations that govern its monitoring and 
providers’ implementation of YAIP. DYCD used contracts from another of its youth programs, 
the Out-of-School-Youth Program (OSY), as a model for YAIP’s perfomiance standards, 
adapting them to YAIP’s target population, which is considered more advantaged than that of 
OSY. Providers that exhibit poor perfonnance on these measures risk the termination of their 
contracts and put their Vendor Infonnation Exchange System (VENDEX) rating in jeopardy. 3 

The first YAIP performance goal is to place 100 percent of participants who complete 
orientation (Phase 1) into an internship. Those who are selected for the program without being 
de-enrolled during the orientation phase — whether because they found employment, had poor 
attendance, or for some other reason — are thereafter considered “enrolled” YAIP participants. 


3 VENDEX is a computerized data system that the New York City government uses to assess a provider’s 
business integrity and capacity to fully perform contract requirements, among other considerations, to justify 
the award of public tax dollars. 
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Figure 3.1 


YAIP Management and Typical Staffing Structure 
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Blue boxes represent DYCD staff. 


Aqua boxes represent provider staff. 


The organizational structure depicted by the blue and aqua boxes represents 1 of the 12 provider agencies. (Note that 1 of these agencies delivers the program in 2 
separate locations, resulting in a total of 13 locations.) The configuration of line staff — or the bottom rung of the chart — varies by provider agency, and this chart 
presents an example of a typical configuration. DYCD requires each provider to fill the following capacities: recruitment, worksite development, case management, 
retention services, educational services, and data entry. On average, these roles are filled mainly by five hill-time staff members (including the Program Coordinator), as 
shown in the bottom two rungs of the organizational chart. 














The second major goal is for at least 75 percent of enrolled participants to complete the intern- 
ship (Phase 2). Internship completion is defined as 245 hours of participation in the program. 
The third performance metric is for at least 70 percent of enrolled participants to be engaged in 
one of four DYCD-approved outcomes after the internship ends: advanced training (a home 
health aide or security guard certification program, for example), unsubsidized employment, 
education, or the military. The final performance expectation is for at least 60 percent of 
enrolled participants to be engaged in a DYCD-approved outcome by the end of the follow-up 
period (Phase 3), which is nine months after the internship ends. 

DYCD’s deadlines, policies, and reporting requirements call for staff who are both de- 
tail oriented and skilled multitaskers. YAIP staff must also be able to develop strong relation- 
ships with participants and excel at program administration. Recruiting staff who meet all of 
these criteria is a significant challenge. Additionally, many YAIP staff members reported 
feeling frustrated by the rigidity of the perfomiance measures. For example, they thought that 
DYCD should recognize providers for engaging participants in more than one placement at the 
same time, such as a participant who enrolls in school while working in an unsubsidized job. 
Under the current system, DYCD counts such dual placements only once toward some program 
performance measures. 4 Moreover, provider staff felt that the CEO and DYCD policy of only 
counting the placements of participants who complete their internships toward perfomiance 
measures rewards internship completion at the cost of other positive outcomes. DYCD estab- 
lished this policy to help ensure that YAIP participants receive the maximum benefit from the 
program and to prevent YAIP providers from prioritizing job placement at the expense of 
YAIP’s other program objectives. However, it leaves staff with a difficult choice: If, for 
example, a participant is offered a job before completing the internship, staff must either advise 
the participant to take the job, which may be in the best interest of the participant, or advise the 
participant to complete the internship, which would allow the provider to receive credit for the 
placement. 5 

YAIP staff are pulled in many directions at once and must divide their time and atten- 
tion across three different cohorts in different stages of the program. With only two weeks 
between the conclusion of one cohort’s internship phase and the start of the next cohort’s 
orientation phase, staff have very little time to reflect on the past cohort and make adjustments 
for the next. Managers and line staff agree that working for YAIP is not a nine-to-five office 


4 For example, if a participant attends classes at a community college and works part time in an unsubsi- 
dized position, the individual is counted in both the education and employment placement categories. Howev- 
er, for the purposes of calculating performance metrics indicating the percentage of participants in a placement, 
the individual is counted only once. 

5 In response to these concerns, CEO and DYCD have revised this policy: the program now reviews early 
placements regardless of internship completion. 
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job; staff work on weekends and evenings, meeting with participants in their homes and 
supporting them in times of crisis. According to many staff members, working at YAIP is akin 
to being part of a family, which illustrates the high level of personal commitment that character- 
ize YAIP staff. Even though YAIP targets the most job-ready disconnected youth, many staff 
members believe that YAIP participants need more support than the program model affords. 
One suggestion that came up in staff interviews is to extend the internship phase and run two 
program cycles per year instead of three. This change would give more time to both participants 
to develop soft and technical skills and staff to reassess and improve upon past performance 
between cohorts. 


Implementation of Core Program Components 

This section describes the implementation of the core program components in greater detail. 
The research team collected data on YAIP’s implementation from each individual provider 
agency, but, because providers delivered the program very similarly, the analysis presented in 
this section examines the program as a whole. However, some provider-specific examples are 
included for illustrative purposes. 

• Overall, the implementation of YAIP aligns closely with the program 
model and is consistent across providers. 

Orientation 

Daily group orientation sessions facilitated by staff at their program offices occupied 
the first two to four weeks of each YAIP cycle. The program pays participants the minimum 
wage for five hours per day, five days per week during this phase. The overarching goals of 
orientation are to prepare participants to succeed in the workplace, lay a foundation for cohort 
cohesion, and match participants to a worksite. The Young Adult Career Development Curricu- 
lum, developed for DYCD and CEO by the Workforce Professionals Training Institute, serves 
as the blueprint for orientation as well as the subsequent weekly educational workshops. 6 The 
curriculum addresses the first two orientation goals of work readiness and cohort building. Each 
module begins with an icebreaker or teambuilding exercise, which accelerates cohort cohesion 
and allows staff to leam more about their participants, including how they interact with each 
other. See Box 3.1 for a complete list of the modules included in this curriculum. 


6 Workforce Professionals Training Institute is a nonprofit training and technical assistance organization 
devoted to strengthening the field of workforce development. 
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Box 3.1 


Young Adult Career Development Curriculum Modules 

Program introduction: program overview, norm setting, motivation to attend 
Personal assessment: personality assessments and goal setting 
Career planning: potential occupations and career pathways 
Understanding employers: traits desired and avoided by employers 
Applications and resumes: writing effective resumes and cover letters 
Interviewing: interview skills and mock interviews 
Finding job openings: online job search and networking techniques 

Staying motivated for the job search: organizational and motivational tools for the job 
search 

Success in the workplace: conflict resolution and workplace behavioral expectations 
Time management: time management scenarios and techniques 

Professional communication skills: phone and e-mail etiquette, public speaking, and custom- 
er service 

“Moving On Up”: leveraging supervisory and constructive feedback, identifying advance- 
ment opportunities, entrepreneurism 

Healthy living: nutrition, exercise, drugs, and sexual health 

Financial literacy: budgeting, bank accounts, credit cards, and financial aid for college 

Create a company competition: participants work in teams to identify, develop, and propose 
a business opportunity 

Providers are required to cover all of the topics in the Young Adult Career Develop- 
ment Curriculum, but do not have to adhere to the curriculum exactly. DYCD and CEO 
encourage providers to treat the curriculum as a template from which to build lesson plans. 

• The degree to which each provider customized the Young Adult Career 
Development Curriculum varied, but all providers covered the core top- 
ics during either orientation or the weekly educational workshops. 
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Several providers reported that parts of the curriculum were too childish, too “corny,” 
or addressed certain topics prematurely for their participants, so they modified the curriculum 
accordingly. One common source of variation among providers is the guest speakers they invite 
to their workshops. These speakers may include employers, former participants, health educa- 
tors, and college representatives, among others. Providers also vary in tenns of when they 
deliver the curriculum. For example, Opportunities for a Better Tomorrow covers the majority 
of the curriculum during orientation and implements its own curriculum for subsequent work- 
shops. The Door, on the other hand, developed its own curriculum for orientation that focuses 
on fostering cohort cohesion and making sure participants feel safe and supported. They 
encourage youth to “come out of their shells” and “take pride in who they are.” For instance, 
participants are asked to stand up and share 15 things about themselves, such as their favorite 
color and middle names, with the goal of making youth more comfortable speaking about 
themselves, something they will likely have to do during job interviews. 

The third overarching goal of orientation is to match participants with internship place- 
ments. The extensive assessment process leading up to enrollment gives staff a solid head start 
on this task, but orientation provides them with an opportunity to get to know participants better 
and to observe their conduct. According to staff, orientation is important because it offers a 
preview of how participants will behave at their worksites. Additionally, staff noted that 
participants may comport themselves differently once they have already been accepted into the 
program and thus feel less pressure to impress staff, or issues such as family obligations or 
behavior problems that were not apparent during intake may present themselves. Staff consider 
participants’ conduct during orientation as they match them with appropriate internship place- 
ments. 


Based on participation data from YAIP’s management infonnation system, 19 percent 
of program group members did not return to the program after random assigmnent (evidenced 
by the 81 percent who attended orientation). (See Table 3.2.) Another 2 percent failed to 
complete orientation and therefore did not continue on to the internship phase, while an addi- 
tional 3 percent completed orientation but exited the program before beginning their internship, 
leaving a total of 77 percent of YAIP program group members who ever worked in an intern- 
ship. 7 Whether the remaining 23 percent of the program group faced significant barriers to 
employment that interfered with their ability to participate, found jobs on their own, or did not 
participate because of some other scenario, these attrition figures suggest that YAIP’s extensive 


7 Based on the very low level of drop-off between orientation completion and internship placement, YAIP 
providers came quite close to meeting the first DYCD performance goal of placing 100 percent of orientation 
completers in an internship. 
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Table 3.2 


One-Year Participation in YAIP Subsidized Internships and Services 
Among Program Group Members 

Measure Program Group 

Attended orientation (%) 81.3 

Completed orientation (%) 79.2 

Worked in a subsidized internship (%) 76.6 

Completed a subsidized internship (%) 65.5 

Sample size 1,638 

Among those who worked in a subsidized internship 

Completed internship (%) 85.6 

Hired into unsubsidized position by internship employer (%) 16.4 

Engaged in outcome placement at end of internship 3 (%) 51.4 

Unsubsidized employment 39.6 

Education 10.1 

Training 3.0 

Miltary service 0.0 

Received case management services during nine-month follow-up period* 3 (%) 85.8 

Average number of contacts during nine-month follow-up period* 3 5.2 

Average number of in-person meetings* 3 1 .4 

Average number of other contacts* 3 3.8 

Engaged in outcome placement at end of nine-month follow-up period 3 (%) 48.5 

Unsubsidized employment 38.8 

Education 9.2 

Training 1.9 

Miltary service 0.2 

Engaged in outcome placement at end of internship and end of nine-month follow-up period (%) 36.5 

Hired by internship employer and engaged in unsubsidized employment at end of nine-month 13.2 

follow-up period (%) 

Sample size 1,254 

SOURCE: MDRC calculations using data from the YAIP management information system. 

NOTES: “Participants may have been engaged in more than one placement type; therefore, the percent of those 
engaged in each placement type may sum to more than 100 percent. 

b Contacts were determined based on a search, using statistical software, of words included in the case notes 
written by YAIP case managers. As a result, these figures should be considered estimates. 
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pre-orientation assessment process — parts of which were modified to meet the requirements of 
the study — does not completely screen out young adults who are not appropriate for the 
program. 

Still, YAIP’s overall participation rate was comparable to similar programs that serve 
disconnected young people, and retention throughout the duration of the program was higher. 
Like YAIP, Project Rise engaged discomiected young people in an education-focused, multi- 
week pre-internship phase, followed by a paid internship. Although 91 percent of enrolled 
program group members began the pre-internship phase (compared with 81 percent of the 
YAIP program group), only 68 percent of enrollees began internships (compared with 77 
percent for YAIP). Similarly, in DYCD’s Summer Youth Employment Program (SYEP) — a 
paid summer internship program for New York City youth — just 67 percent of first-time 
applicants selected for the program in a lottery ever worked in an internship, which is 10 
percentage points lower than YAIP’s internship participation rate. 8 While neither of these 
programs aligns perfectly with YAIP, these figures indicate that YAIP’s retention rate after the 
start of the program compares favorably with other interventions targeting disconnected youth, 
though its initial participation rate during the evaluation was slightly lower. 

An additional advantage of beginning the internship selection and placement process 
during orientation is that it complements the “world of work” skills taught in the Young Adult 
Career Development Curriculum. As participants learn about different careers and how to job 
search, YAIP providers present them with a diverse list of worksites from which to choose. 9 
Likewise, participants can put their newly developed interviewing skills to use when they 
interview for their internships, though some worksites do not require an interview and accept 
whomever the program refers. 

• The process of matching YAIP participants to internships involves 
much discussion and collaboration between provider staff, participants, 
and worksites to ensure the right fit based on participants’ strengths, in- 
terests, and goals, as well as the worksites’ environment, culture, and 
needs. 

The internship matching process is collaborative, with YAIP provider staff and partici- 
pants meeting one-on-one to discuss the participant’s strengths, interests, and goals. Staff 
consider all of these dimensions, as well as participants’ personalities and the environment and 

8 The SYEP evaluation analyzed data from years 2006 to 2010, and the participation rate was calculated 
among the young people selected in the lottery in the first year they applied to the program. See Valentine, 
Anderson, Hossain, and Unterman (2017) for additional information. 

9 Different YAIP providers rarely share the same worksites: 92 percent of worksites are unique to each 
provider. 
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culture at a worksite, during the matching process. Most providers also administer some sort of 
personality assessment tool, such as the Myers-Briggs Type Indicator, and a professional 
assessment tool, such as CareerZone, to help facilitate conversations with participants. 10 In 
addition to one-on-one conversations with participants, staff confer with one other and with the 
worksites to ensure the right fit. 

Some providers invite worksite representatives to orientation to give them an opportuni- 
ty to meet with all of the participants and to give participants a chance to leam more about the 
internships offered. Some providers supply participants with a list of worksites (for example, 
one site provided participants with a list of 13 worksites from which to choose, each with one to 
five internship slots) and a brief description of the internships available and ask the participants 
to rank their first few choices. Overall, provider staff reported trying to accommodate partici- 
pants’ preferences and usually being able to do so. 

Internship 

At the heart of YAIP is the paid internship, which promotes the program goals in direct 
ways, such as providing participants with employment and income, and in indirect ways, such 
as exposing participants to novel workplace experiences. The internship serves as a sort of dress 
rehearsal in which participants can practice the skills taught during orientation and subsequent 
educational workshops. Since the internship experience is subsidized (DYCD pays the partici- 
pants’ wages and employers do not have to commit to hiring participants), the environment 
allows participants some room to make and leam from workplace mistakes. Furthermore, 
internships expose participants to a job category, work environment, industry, and so on, giving 
them firsthand experience that helps them determine what career path they may want to pursue 
in the future. One provider shared an anecdote about a participant who said he enjoyed cooking 
and was therefore placed in an internship in which he could earn a food handlers certificate. 
However, he also worked maintenance during his internship and ultimately decided that he 
wanted to pursue a career in building maintenance instead of food service. At the very least, all 
internships provide participants with recent work history to bolster their resumes. 

Several overarching worksite characteristics factor into providers’ worksite develop- 
ment efforts. On the whole, providers tend to value worksites that demonstrate one or more of 
the following characteristics: 

• Commitment to developing participants. This commitment refers mainly to 
worksite supervisors who understand that YAIP participants have a lot to 


10 CareerZone is a website maintained by the New York State Department of Labor whose purpose is to 
help people explore different career paths. 
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learn and are willing to take the time to guide them. Despite provider staff 
stressing the importance of hands-on supervision, just over half of YAIP par- 
ticipants who completed a participant questionnaire agreed strongly that they 
felt supported by their supervisors (Figure 3.2). On the other hand, about 
three-fourths of respondents strongly agreed that they were learning im- 
portant soft skills from their internships, which aligns with worksite supervi- 
sor responses to a separate employer or worksite supervisor questionnaire in 
which 82 percent of respondents strongly agreed that part of their role as 
YAIP supervisors is to help participants learn soft skills (Appendix Table 
B. 1). 11 In other words, participants believed they were learning important les- 
sons from their worksite supervisors and supervisors understood that teach- 
ing these lessons was part of their role, but many participants would have 
preferred additional support. Interviews with worksite partners also revealed 
altruistic motivations behind participating in YAIP: Several supervisors de- 
scribed the gratification that comes from coaching YAIP interns. 

• Possibility of hire. Some internships serve as a trial work period that may 
lead to unsubsidized employment. For worksites with the capacity and need 
for new employees, there is an expectation that they will hire high- 
performing participants after the internship. (However, the employer is under 
no obligation to do so.) As shown in Table 3.2, internship employers hired 16 
percent of program group members, with the majority of hires happening in 
the social service; retail; and professional, legal, and financial services indus- 
tries. Interestingly, 41 percent of employers who responded to the employer 
or supervisor questionnaire stated that one of the reasons they participated in 
YAIP was to test new workers without risk but with the potential to hire later 
(Appendix Table B.l). 

• Unique work experiences. A small number of internships provide a unique 
experience that participants are unlikely to obtain without the program’s 
connections. For instance, some YAIP participants intern at Goldman Sachs; 
the Intrepid Sea, Air, and Space Museum; and the Children’s Museum of the 


"it is important to note the limitations of both the participant and employer or worksite supervisor ques- 
tionnaire results. An incomplete sample of participants (approximately 60 to 65 percent) and employers or 
worksite supervisors (approximately 10 to 15 percent) responded to their respective questionnaires. Thus, the 
data collected may not be representative of the views of all participants or worksite personnel. 
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Figure 3.2 


YAIP Participants with a Favorable Impression of 
Internship Support and Preparation for Future Employment 
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Sample size = 570 


SOURCE: MDRC calculations from the Subsidized and Transtitional Employment Demonstration youth 
participant questionnaire. 


NOTES: The four measures presented in this figure were created based on an exploratory factor analysis of a 
pool of questions. These questions asked participants about their level of agreement with a particular statement 
on a scale of 1 to 7, where 1 indicates strong disagreement and 7 indicates strong agreement. Based on the 
results of the factor analysis, questions were grouped into factors and a mean score was calculated across the 
questions included in a particular factor; the percentages presented above represent the proportion of 
questionnaire respondents who averaged a score of 6 or higher on the questions in that factor, indicating a high 
level of satisfaction with their program experiences in that area. 

Questionnaires were administered to participants during weekly educational workshops at each provider, 
when many participants would be available at once. Consequently, the responses obtained are from participants 
who attended educational workshops and are therefore likely to have been more motivated and engaged than the 
full sample of program participants. For this reason, the results presented in this figure are not necessarily 
representative of all participant experiences and should be interpreted with caution; they are likely to be more 
positive. 

a Measure is based on agreement with the following statements: I understand what is expected of me on the 
job ; I know whom at work to ask for help when I need it, my relationships with coworkers are positive and 
supportive', and my coworkers understand me and want me to succeed. 

b Measure is based on agreement with the following statements: I get the support or guidance that I need from 
my supeiwisor, my superx’isor gives me advice about how to handle situations at work', and my supervisor helps 
me if personal issues come up that get in the way of working. 


(continued) 
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Figure 3.2 (continued) 


NOTES (continued): 

c Measure is based on agreement with the following statements: I am learning how to work better with 
coworkers', I am learning how to cooperate better with supervisors', and this job has helped me learn to present 
myself better at work. 

d Measure is based on agreement with the following statements: The kind of work I am doing will help me get a 
decent-paying job later, I am learning specific job skills that I will use in the future', and 1 have met people through 
this job who may help me find a job in the future. 


Arts. The Children’s Museum of the Arts is an art education facility in Man- 
hattan where YAIP interns assist artists in facilitating workshops for young 
children. The internship culminates with a workshop that the intern plans and 
leads. 

Each provider is required to maintain a portfolio of worksites that contains at least 50 
work slots across several worksites (no more than five slots per worksite). At the time of the 
study, each provider offered participants a selection of internship placements at between 12 and 
29 worksites. As shown in Table 3.3, 51 percent of worksites were in the private or for-profit 
sector, 42 percent were nonprofits, and about 8 percent were government entities. 12 Table 3.3 
also shows the breakdown of worksites by industry. 

To host a YAIP intern, worksites must submit an application that describes the work- 
place, the supervision that they will provide to the intern, and the tasks the intern is expected to 
perform. It is difficult to generalize about the nature of supervision at worksites because of their 
number and diversity. One aspect that is consistent across worksites and providers, however, is 
the mid- and end-of-cycle evaluations. Worksite supervisors rate each participant on a scale 
from one (unsatisfactory) to five (excellent) on 10 dimensions at the middle and end of the 
internship; Box 3.2 describes these dimensions. YAIP staff review these evaluations, which 
provides them with a way to monitor what is happening at the worksites and what they need to 
work on with participants. 

YAIP participants are on DYCD’s payroll; thus, time sheet collection offers another 
formal contact point between YAIP providers and worksites. Worksite supervisors complete the 
timesheets and program staff review and submit them to DYCD. This procedure ensures that 


12 Among the nonprofit worksites, 49 percent were social service agencies, 15 percent were educational 
service agencies, and 9 percent were health care-related agencies. The remaining 27 percent of nonprofit 
worksites were scattered among 15 other industries. 


45 



Table 3.3 


YAIP Employer Characteristics 


Measure 

YAIP 

Employers 

Employer type (%) 

Private/for-profit 

50.6 

Nonprofit 

41.8 

Government 

7.6 

Employer industry (%) 

Accommodation and food services 

5.3 

Arts, entertainment, and recreation 

9.4 

Community or social services 

21.5 

Construction, manufacturing, transportation, and utilities services 

3.5 

Day care or day camp 

4.4 

Educational services 

9.1 

Health care 

8.5 

Professional, legal, and financial services 

10.0 

Public administration and government 

5.0 

Real estate and property 

2.9 

Retail 

17.9 

Other 

2.4 

Average number of participants per employer 11 

4.8 

Number of participants per employer 11 (%) 

1 participant 

28.2 

2-3 participants 

28.5 

4-6 participants 

21.5 

7 or more participants 

21.8 

Sample size 

340 


SOURCE: MDRC calculations using data from the YAIP management information system. 

NOTES: a Employers may have taken on participants from YAIP providers not participating in the evaluation. 
These participants are included in these measures. 
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Box 3.2 


YAIP Participant Mid- and End-of-Cycle Evaluation Dimensions 

Work habits: displays a positive, cooperative attitude toward tasks and work assignments 

Communication skills: expresses ideas clearly, both verbally and in writing; listens well; 
responds appropriately to workplace requests 

Dependability: understands expectations of timeliness for task completion; adheres to 
timeframes and is punctual in completing assignments 

Cooperation: works well with coworkers and supervisors; demonstrates consideration, 
rapport, and helpfulness 

Initiative: seeks and assumes greater responsibilities; seeks out information to improve skill s 
and performance 

Adaptability: adjusts to changes in tasks and responsibilities; accepts new ideas and responds 
appropriately to constructive criticism; completes projects fully even when undesirable or 
unpleasant 

Judgment: effectively analyzes problems; determines appropriate, timely, and decisive 
actions; thin ks logically; resolves conflicts with persons of authority, coworkers, and custom- 
ers in the appropriate manner 

Attendance and punctuality: considers number of absences, tardiness, or both; calls supervi- 
sor when absent or late; takes and returns from breaks as scheduled 

Planning and organizing: plans, organizes, and carries out assignments; coordinates with 
others; establishes priorities; demonstrates effective time management 

Leadership skills: demonstrates effective leadership abilities; earns respect and cooperation; 
inspires and motivates peers 


staff visit and communicate with worksites regularly. However, many program staff find the 
timesheet procedure to be onerous. Because DYCD uses hardcopy timesheets, staff must travel 
to all of the worksites every other week to collect them. This task is typically the responsibility 
of one to three staff members, but for staff who are already stretched thin, and with an average 
of 25 different worksites dispersed throughout the city, it is inconvenient and time consuming. 

Aside from these biweekly site visits, most communication with worksites occurs as- 
needed and may be initiated by the worksite or the YAIP provider, depending on the issue. 
Worksite personnel who responded to the employer or worksite supervisor questionnaire 
reported high levels of satisfaction with providers’ responsiveness and effectiveness in handling 
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problem situations. However, as shown in Appendix Table B.l, only about 43 percent of 
respondents actually reported using the YAIP provider as a resource to help resolve issues with 
participants. 

While the worksites that host YAIP interns represent a wide range of industries and sec- 
tors, the job responsibilities of most all of the internships cluster around a few common tasks. 
According to participants who responded to the questionnaire (Appendix Table B.2), the tasks 
that most participants typically perform at their internships include clerical work (filing, making 
copies, and answering phones) (52 percent), computer-based work (49 percent), interacting with 
customers (48 percent), and maintenance work (45 percent). 

• More than three-fourths of the YAIP program group worked in a subsi- 
dized internship. Among this group, 86 percent completed their intern- 
ship. 

As shown in Table 3.2, 77 percent of program group members worked in a subsidized 
internship and, among those who did, the vast majority completed their internships. Among the 
full program group, just under two-thirds completed an internship. It is important to note that 
DYCD’s performance goals call for at least 75 percent of enrolled participants to complete their 
internships. Among enrolled participants in the three study cohorts, 83 percent completed an 
internship, exceeding the DYCD benchmark (not shown in table). 

Participants earn minimum wage for working up to 20 hours per week at their intern- 
ships; they also earn minimum wage for attending the program’s weekly five-hour educational 
workshops. DYCD disburses wages electronically to a debit card issued to each participant at 
the beginning of the program. Participants are paid every other week for the hours documented 
in their timesheets. As shown in Figure 3.3, which depicts the average participation timeline 
during Phases 1 and 2 for participants who worked in a subsidized internship, this subset of 
participants averaged over nine weeks in their internships and participated in program activities 
for an average total of 22 hours per week, including both work in their internships and attend- 
ance at educational workshops. 13 

Participants’ work schedules depend on the particular worksite, and the worksites that 
offer flexible hours are the most sought after by participants who must balance their internships 
with their roles as parents, caregivers, or students. For example, the worksite supervisor at a 
wholesale hardware and building supply store reported that he adjusts the interns’ schedules — 


13 Due to data limitations, it is not possible to calculate the hours participants worked in an internship 
placement separately from the hours they spent in educational workshops. 
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Figure 3.3 


Average YAIP Participation Timeline Among Program Group 
Members Who Worked in a Subsidized Internship 


Orientation starts: 
Day 8 



Five-day 0 j 2 3 4 5 6 7 8 9 10 11 12 13 14 15 

workweek i i 


22 hours worked per week during 
internship 13 

Sample size = 1,223 

SOURCE: MDRC calculations using data from the YAIP management information system. 

NOTES: This figure is based on the average participation of program group members who worked in a subsidized 
internship and assumes a five-day workweek. Each measure excludes 34 participants who participated in a cohort 
that was later than the one in which they were randomly assigned. 

“This measure was calculated using total hours worked assuming a five-hour workday. It includes educational 
workshops, which were five hours per week. 

b This measure was calculated using weeks with greater than zero hours worked. It includes educational 
workshops, which were five hours per week. 


allowing them to arrive and depart earlier — to accommodate night school. Similarly, the 
worksite supervisor at a children’s art museum allows interns to work on weekends if they need 
to miss their regular weekday hours due to other responsibilities. 
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• YAIP staff, worksite supervisors, and participants seem to agree that the 
internship is a positive learning experience for participants, but may not 
sufficiently enhance their long-term employability. 

Many staff believe that a longer internship is needed to further develop participants’ 
skills. As shown in Appendix Table B.l, 62 percent of employer or worksite supervisor ques- 
tionnaire respondents indicated that YAIP participants are less prepared than typical entry-level 
workers on their first day of work. However, 70 percent reported that, over the course of the 
internship, YAIP participants also demonstrate greater gains than the typical entry-level worker. 
Figure 3.2 shows that participants are also aware of the internship’s limited impact, with only 58 
percent strongly agreeing that their internship experience will improve their Iliture employment 
prospects. 

Educational Workshops 

Once participants are placed in their internships, they return to the program offices once 
per week, as a cohort, for mandatory five-hour educational workshops. 14 Providers view 
workshops as an opportunity to delve deeper into the same topics that they present only briefly 
during orientation. As they do with orientation, YAIP staff use the Young Adult Career Devel- 
opment Curriculum (described in Box 3.1) as a guide for the weekly workshops. Staff may also 
supplement the curriculum with field trips, such as college tours or a trip to one of the city’s 
botanical gardens, as another tool to expand participants’ horizons. 15 

• Though participants are paid for attending mandatory weekly educa- 
tional workshops, providers reported that workshop attendance was a 
challenge. 

Another objective of the weekly workshops is to bring the cohort back together, allow- 
ing the group to continue to gel and making it easier for staff to check in with participants. 
Participants intern at different worksites, many of them employed for the first time, so it is 
valuable for them to be able to come together to share their experiences. Participants’ real-life 
internship experiences enrich work-readiness lessons. For example, workshop facilitators may 
present the cohort with a scenario, such as a disagreement with a supervisor, that someone in the 
cohort experienced and ask participants to discuss possible resolutions to the situation. Despite 


14 Some providers split the cohort into two different groups and run the same workshop with each group on 
different days. Providers who employed this strategy reported that it made the groups more manageable. 

15 Multiple providers complained that the process of getting these trips approved by DY CD — whether due 
to timing constraints or DYCD’s assessment of their appropriateness — was a significant deterrent to organiz- 
ing them. 
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the fact that these workshops did not conflict with participants’ work schedules and that 
participants are paid to attend them, providers reported that attendance was often low. 

Many providers offer incentives for exemplary workshop attendance, such as treating 
participants to dinner or giving them movie tickets. Unfortunately, due to data limitations, it is 
not possible to provide exact attendance rates based on data from YAIP’s management infor- 
mation system. 

Case Management 

YAIP providers infuse case management throughout the program in formal and infor- 
mal ways, in large part because case managers are involved in most if not all aspects of the 
program, from recruitment through the follow-up phase. Through consistent and frequent 
informal interactions, case managers develop an almost familial bond with participants. Staff 
members reported that a caring and trusting relationship such as this one is key to the program’s 
success because it increases the likelihood that a participant will be receptive to the advice and 
life lessons that staff members offer. Similarly, if a participant has such a relationship, the 
young person is more likely to ask staff for help when it is needed, making the task of case 
management easier. 

YAIP’s formal case management component centers on the Individual Service Strategy 
Form. The form is a working document used by staff and participants to identify, monitor, and 
evaluate a participant’s strengths, interests, and goals on an ongoing basis. Case managers 
complete the form over the course of the program during one-on-one meetings with partici- 
pants. Each section of the fomi corresponds to different milestones in the program. They 
document participants’ strengths, interests, and skills following orientation. Case managers 
review participants’ mid-cycle worksite evaluations and self-evaluations halfway through the 
program, and fonnulate post-internship plans with participants between the seventh and tenth 
week. Finally, they review participants’ end-of-cycle worksite evaluations and self-evaluations 
in the last weeks of the internship phase. The continuous assessment of goals and progress is 
critical for disconnected young adults. Staff reported that participants frequently change their 
minds about their goals and plans for the future. For example, they may apply to YAIP with the 
intention of entering the workforce only to realize that they want to go back to school to pursue 
a different career. Staff also explained that many participants struggle with long-term planning 
because they overlook the interim steps needed to reach the ultimate goal. By periodically 
revisiting participants’ objectives, staff can help young adults with ambitious goals to stay on 
track to achieving them. 

Staff also work with participants to address issues that may interfere with their attend- 
ance at their internships. According to the participant questionnaire (Appendix Table B.2), some 
of the most common services participants need are help managing money (58 percent of 
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respondents), help obtaining work-appropriate clothing (47 percent of respondents), and help 
with transportation (37 percent of respondents). Some programs are better positioned than 
others to connect their participants to needed services, typically through streamlined referrals to 
partner organizations or even in-house referrals. Interestingly, YAIP programs operating within 
multiservice agencies do not always benefit from the wide array of services offered, in part 
because of varying eligibility criteria and in part because agencies may offer these services at 
different locations from where they offer YAIP. Staff reported that participants most commonly 
encounter mental health and housing challenges, problems that staff frequently do not have 
adequate resources to address. 

• YAIP case managers sought to balance the urge to provide intensive 
support to participants and the desire to foster self-sufficiency. 

Case management can be somewhat of a balancing act in YAIP. Many staff members 
go above and beyond their job descriptions to help participants, working outside of normal 
business hours and coming to participants’ aid in times of crisis. YAIP’s case managers serve as 
the caring, reliable adults whom many participants lack in their lives. As one case manager put 
it, “so many kids have such dysfunctional homes that nobody is paying attention to them. We 
are an extra hand for them.” On the other hand, staff try to avoid what they see as “babying” 
their participants in order to foster their independence and resourcefulness. Another case 
manager explained that while staff will impart information about available services, it is “not on 
us” to follow up on the referral. In other words, YAIP’s case managers aim to provide partici- 
pants with the tools to be self-sufficient. 

Outcome Placements and Follow-Up Services 

Case managers begin formulating post-internship plans with their participants about 
hallway through the program. As shown in Table 3.2, a little more than half of those who 
worked in an internship achieved one of the DYCD-approved outcome placements at the 
conclusion of the internship period. Fifty percent of enrolled participants did so, falling short of 
DYCD’s perfonnance benchmark of 70 percent (not shown in table). 16 

• About half of participants who worked in an internship were engaged in 
a DYCD-approved outcome placement at the end of the internship peri- 
od; the same was true at the end of the nine-month follow-up period. 
Placements in unsubsidized employment were most common. 


16 The proportion of participants who achieved a DYCD-approved outcome placement is higher (59 per- 
cent) among those who completed their internships. 17 A contact was counted as any time a case manager was 
able to successfully reach a participant, including phone, social media, and in-person contact. 
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While YAIP is primarily a workforce development program, the program model 
acknowledges that labor market demand is greater for workers with at least a high school 
diploma or equivalency certificate and preferably some college or vocational training. Indeed, 
many staff believe that their participants should continue their education, but securing employ- 
ment often takes priority because many cannot afford school and have a more immediate need 
for income. Consequently, placements in unsubsidized employment (40 percent of participants 
who worked in an internship) vastly outpaced education placements (10 percent of participants 
who worked in an internship). For participants who expressed an interest in education (45 
percent of participant questionnaire respondents indicated that they had decided to go back to 
school), staff try to help them figure out how to manage attending school while working. 

Advanced training is a relatively easy outcome for participants to achieve because ap- 
plication processes for training programs tend to be more straightforward than the college 
application process, for example, and the required commitment is generally shorter tenn 
compared with enlisting in the military, for instance. However, some providers caution partici- 
pants against spending time and money on training certifications that do not necessarily im- 
prove their employability, which may explain why only 3 percent of participants enrolled in 
training after their internships. Military enlistment, the last DYCD-approved program outcome, 
was rare: No program group members enlisted in the military at the end of the internship, and 
only 0.2 percent of program group members were enlisted at the end of the nine-month follow- 
up period. 

Whether or not participants were in a placement at the end of their internship, YAIP 
staff continued to provide follow-up support services to them over the next nine months. These 
nine months make up Phase 3, or the follow-up phase, when providers focus on helping 
participants secure or sustain a placement. This follow-up period is the longest yet least inten- 
sive phase of the program, when staff work with participants mainly on an individual basis. 
During this phase, provider staff try to maintain contact with participants in order to assist them 
with job leads, verify their outcome placements, help connect them to needed services, and 
provide other case management and counseling support. The participation data presented in 
Table 3.2 align with this description of services: 86 percent of participants who worked in an 
internship received case management during the nine-month follow-up, with an average of five 
contacts from YAIP staff. 17 Motivated by their relationships with staff, their cohort, or both, 
some participants stay in touch of their own volition. Other participants require a great deal of 
persistence on the part of staff, who use any outlet available to them — phone calls, friends, 
family, home visits, social media — to check in with participants. 


17 A contact was counted as any time a case manager was able to successfully reach a participant, including 
phone, social media, and in-person contact. 


53 



In the study sample, 49 percent of participants who worked in an internship were en- 
gaged in an outcome placement at the nine-month mark, with a similar breakdown among 
placement types at the close of the internship period. However, an important caveat to this 
figure is that YAIP providers must submit verification of outcome placements, such as an 
official letter or paystub, in order to receive credit for the placement from DYCD. Given the 
difficulties of obtaining verification, it is possible that the proportion of participants engaged in 
an outcome placement at the nine-month mark is understated. These difficulties include the 
program likely losing touch with a number of participants during the nine-month follow-up 
period (about 14 percent of participants who worked in a subsidized internship did not receive 
case management services during the follow-up period) and still other participants likely not 
providing documentation of their placements. 

The last of YAIP’ s performance expectations is to retain at least 60 percent of enrolled 
participants in a program-approved outcome at the end of the follow-up period. The proportion 
of enrolled participants in an outcome placement at the nine-month mark was about 47 percent 
(not shown in Table 3.2). This figure again falls short of DYCD’s perfonnance benchmark. 

There are two measures in Table 3.2 that help shed light on the extent of placement 
continuity. The proportion of internship participants engaged in an outcome placement at the 
end of their internship and at the end of the follow-up period (37 percent) suggests that partici- 
pants who were engaged in a placement after completing their internships were much more 
likely to be engaged in such a placement nine months later. Similarly, the 13 percent of partici- 
pants hired by their worksites at the close of their internship and working in an unsubsidized job 
at the end of the follow-up period indicates that participants hired by their worksites (16 
percent) were likely to still be employed nine months later. While these continuity measures do 
not capture any potential disconnection which may have occurred between the two times when 
placement was measured, these data points nevertheless highlight the importance of a partici- 
pant’s status at the end of the internship phase. 

Boxes 3.3 and 3.4 offer in-depth descriptions of two different YAIP participants and 
their respective experiences in the program. These short vignettes, though anonymized, reflect 
actual participant experiences and may help to better illustrate how disconnected young people 
interacted with YAIP program services. 


Conclusion 

YAIP aims to expand positive social networks by allowing participants to develop connections 
with caring adults (staff), like-minded peers (members of their cohort and fellow interns), and 
people with stable employment (worksite supervisors). Implementation data suggest that 
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Box 3.3 


Participant Vignette: “Tiffany” 

Tiffany learned about YA1P from a career counselor at the high school equivalency (HSE) 
preparation program in which she was previously enrolled. Tiffany was interested in YAIP as 
a means of supporting her pursuit of a college education while working part time. She lived 
with her mother in Queens and, before YAIP, had been looking for a job on her own without 
success. 

Tiffany joined YAIP when she was 18 years old. She had obtained her HSE certificate about 
five months earlier, and had no work experience. YAIP staff described her as “structured and 
disciplined.” Staff also described Tiffany as rebellious; she had several run-ins with the law as 
a teenager, though she was never convicted of a crime. Tiffany herself admits that she hung 
around with peers that were involved with drugs and gangs and were a bad influence on her. 
YAIP staff, on the other hand, attributed Tiffany’s rebelliousness to her mother’s “tough 
parenting,” and they felt reassured that if they encountered problems with Tiffany’s program 
participation, they could call home and get her mother’s support. Such action, however, turned 
out to be unnecessary since Tiffany’s attendance at the internship and workshops was perfect. 

Tiffany was interested in a career in criminal justice, but staff prioritized other factors in 
determining her internship placement. They guided her toward a highly structured worksite 
where she would receive the extra mentoring they believed she needed. Tiffany’s internship 
was in the mailroom of a large publishing company. She felt pressured to accept this place- 
ment and initially wished she were placed in a simple retail job in which she would not have to 
work hard while she was in school. As time passed, however, she learned that she was capable 
of the work required and became aware of her employer’s global connections and renown, 
giving her a sense of pride. There were some bumps along the way that staff were able to help 
smooth over. For example, her worksite e-mailed YAIP staff to complain that Tiffany was not 
checking her e-mails in a timely manner. YAIP staff worked with Tiffany to improve her 
professional communication skills and nurture her confidence. Staff also helped her to apply to 
and enroll in community college. 

At the end of the internship phase, Tiffany’s worksite hired her as a permanent employee. 
About a year after joining YAIP, she was still at the same job, working 40 hours per week at 
$9.50 per hour, which at the time was more than the minimum wage of $8.75. YAIP staff 
stayed in touch with Tiffany throughout the nine-month program follow-up phase, though they 
wished she were more proactive in reaching out to them. 

Tiffany’s social and professional network expanded as a result of her participation in YAIP. 
She befriended fellow interns and coworkers, developed professional relationships with 
supervisors, and gained the support of YAIP staff. Of one staff person, Tiffany said, “...there 
were so many times I didn’t want to do something that she would make me do that wound up 
being for the better.” 
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Box 3.4 


Participant Vignette: “Marvin” 

Marvin learned about YAIP from his brother, who had previously participated in the program. 
Marvin was 24 years old and living with his mother and stepfather in Brooklyn when he 
enrolled in YAIP. He came to the program with entry-level, unskilled work experience in fast 
food, retail, and health care, but had not been employed in more than six months. Marvin also 
had attended some college. His goals were to return to school, find a job, and eventually 
pursue a career in journalism. 

Staff at YAIP saw in Marvin a “smart young man” who was “a bit scattered” and lacked 
sufficient soft skills. They believed he would benefit from interning in a city council member’s 
office, where he would be expected to behave and communicate professionally. Marvin 
initially resisted this placement, but grew to enjoy and value the experience. According to 
Marvin, “this whole office experience is new to me... I’m answering phones in the correct 
manner, doing stuff. ..on the computer... The job helps me. It builds me up more, ‘cause 
there’s some things I was lacking and, doing this job, it’s been better for me.” 

Regarding what he learned from YAIP workshops, Marvin said, “I’ve learned to come out of 
my shell, out of my comfort zone, being able to network with people and how to properly 
introduce myself and properly talk with people.” The program also helped him to apply to two- 
and four-year colleges. While he appreciated the benefits of participating in YAIP, Marvin 
repeatedly expressed a strong “do-it-yourself’ attitude, which may have prevented him from 
taking full advantage of the help available to him from program staff. 

Marvin knew from the outset that his internship would not lead to an unsubsidized position. 
Although he enjoyed the internship, this fact did not bother him because he was not interested 
in a career in politics. Still, he valued the networking skills and opportunities that the internship 
afforded. His attendance was excellent and he felt confident that he could ask his supervisors 
for a strong professional reference. 

Unstable housing had been a major obstacle for Marvin and his family for much of his life. 
Shortly after completing his internship, Marvin went to live with his brother in a neighboring 
state because his parents were priced out of the home he had shared with them in Brooklyn. 
Beyond the reach of the program’s local network, there was little YAIP staff could do to help 
connect him with resources, though they stayed in touch with Marvin over e-mail. At the end 
of the nine-month program follow-up period, Marvin was employed, but the data available did 
not include specific information about his employment. 


participants formed relationships that could potentially help them in the future. The fact that 
86 percent of participants who worked in a subsidized internship received case management 
services in the nine-month period following their internships — when little or no formal 
program services are in place — highlights the staffs ability to connect with their partici- 
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pants. In addition to staff support, participants who completed their internships stand to gain 
a professional reference from their worksite supervisors. 

Some provider staff love YAIP, citing the model’s flexibility in terms of customizing 
the workshop curriculum and allowing participants the option of pursuing employment, educa- 
tion, training, or the military after the internship. At the same time, many staff are highly critical 
of the service model and feel restricted and overwhelmed by DYCD’s performance measures 
and oversight. Providers reported that, as a result of these challenges, there is frequent staff 
turnover, which may adversely affect the quality of service delivery. 

YAIP achieved high participation rates overall and participants reported that they 
gained important “world of work” skills as a result of the program. While the vast majority of 
participants who began an internship completed it, only about half were engaged in unsubsi- 
dized employment, education, or advanced training when their internships ended. Interestingly, 
though YAIP was developed as a “light-touch” intervention to help reengage a mostly job-ready 
subset of the disconnected youth population, many involved with YAIP, including provider 
staff, supervisors at internship sites, and young people themselves, believed that YAIP partici- 
pants require both a higher level of support and a longer intervention to improve their educa- 
tional and labor market outcomes. The overall inability of YAIP providers to meet DYCD’s 
perfonnance benchmarks for outcome placements would seem to support this belief. 

Overall, YAIP providers implemented the program very similarly and with a high de- 
gree of fidelity to the intended program model, although there was some variation as a result of 
particular staff, worksite partnerships, and agency resources. This high level of fidelity to 
DYCD’s and CEO’s model allows for a strong test of the program’s effectiveness. 
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Chapter 4 

The Impacts of YAIP on Youth Outcomes 


As discussed in earlier chapters, the goal of YAIP is to improve the labor market prospects of a 
population of job-ready disconnected young adults in New York City. For this report, the 
available follow-up period to assess outcome differences between program and control group 
members was just over one year after random assignment. Since this follow-up period includes 
the time when many program group members were engaged in internships, the early impacts 
shown here are likely a direct result of these placements and are not necessarily indicative of the 
longer-term effects of the program, which a later report will present. 

The YAIP program model includes placement in a paid internship, supplemented by 
weekly educational workshops, along with the development of supportive peer, staff, and 
mentor relationships. The research team expects the program to have initial, short-term effects 
on employment and earnings; since many program group members participated in paid orienta- 
tion and internship activities lasting about 12 to 14 weeks, they are expected to work more and 
earn more than control group members in the period immediately following random assign- 
ment. 


The follow-up phase of YAIP focuses on supporting participants in achieving an “out- 
come placement” — that is, engagement in unsubsidized employment, education, training, or 
the military. In addition to these outcomes, the research team also examined outcomes in several 
other key domains, including psychosocial outcomes, economic and personal well-being, and 
criminal involvement. In order to assess the impact of YAIP in these various areas, the team 
compared the outcomes of program group members with the outcomes of control group 
members. While individuals in the control group did not receive YAIP services, they were free 
to look for jobs on their own or to seek out and receive assistance from other programs and 
organizations in the community. Thus, it is necessary to assess the differences in types and 
amount of assistance that program and control group members received in order to fully 
understand the two research groups’ potential outcome differences. 


Impacts on Service Receipt 

This section focuses on impacts on the receipt of employment and supportive services and is 
based on sample members’ responses to a survey administered roughly 12 months after random 
assignment. The next section, focused on impacts on employment and earnings, considers 
participation in subsidized employment. 
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• Program group members were more likely to report that they received 
help finding or keeping a job in the year following random assignment. 

As discussed in earlier chapters, several other employment programs in New York City 
provide services to people seeking work, so it is not particularly surprising that over half (53 
percent) of the control group reported receiving help finding or keeping a job in the year 
following random assignment (presented in Table 4.1). (See Box 4.1 for a detailed description 
of how to read the impact tables in this report.) However, 85 percent of program group mem- 
bers reported receiving employment assistance. While the difference between the research 
groups’ outcomes is smaller than that found in other studies of subsidized employment pro- 
grams, in which control group members had access to fewer alternative services, it still repre- 
sents a substantial increase in receipt of employment-related services for program group 
members. 


• Program group members were more likely than control group members 
to report receipt of advice or support from both peers and staff, but the 
largest impacts were for supportive staff relationships. 

Creating supportive relationships for participants is a central component of the YAIP 
model, including both relationships with peers and staff members or mentors. As shown in 
Table 4.1, while most (95 percent) of program group members reported receiving advice or 
support from a peer, 85 percent of the control group also reported similar support from a peer. 
Though this difference is modest, the supportive peer relationships almost universally reported 
by program group members reflects YAIP’s emphasis on developing social networks. 

While both program and control group members less commonly reported receiving 
support, advice, or mentoring from staff, a far greater proportion of program group members 
reported engaging in supportive staff relationships. This impact (or difference between the 
program and control groups) was much larger than the impact for supportive peer relationships. 
This result reflects the implementation finding presented in Chapter 3 that the YAIP cohort 
model allows staff to form relationships and work intensely with a relatively small group of 
participants and provides another avenue to support disconnected young adults. 

While YAIP overall had some positive impacts on program participants’ receipt of ser- 
vices, these impacts were relatively modest since many control group members received 
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Table 4.1 


One- Year Impacts on Service Receipt 


Outcome (%) 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

90 Percent 

Confidence 

Interval 

Employment suDDort 

Received help related to finding or keeping a job 

84.6 

53.3 

31.3 *** 

[28.3,34.4] 

Job search or job readiness 2 

79.2 

43.8 

35 4 *** 

[32.2,38.7] 

Career planning 

76.2 

41.5 

34 7 *** 

[31.3,38.0] 

Unpaid work experience or internship 

18.0 

11.4 

6.6 *** 

[3.9, 9.2] 

Paying for job-related transportation or equipment costs 

Other support and services 

Received advice or support from staff member at an agency 

42.6 

20.4 

22.2 *** 

[18.8,25.6] 

or organization 

70.0 

45.6 

24 4 *** 

[20.9,27.9] 

Received advice or support from peer at an agency 

or organization 

94.7 

85.4 

g 2 *** 

[7.2,11.4] 

Received mentoring from staff member at an agency 

or organization 

62.3 

35.0 

27.2 *** 

[23.7,30.8] 

Received mental health assistance 

9.0 

8.9 

0.0 

[- 2 . 0 , 2 . 1 ] 

Received substance abuse treatment or counseling 

2.0 

1.3 

0.7 

[-0.3, 1.6] 

Sample size (total = 2,127) 

1,327 

800 




SOURCE: MDRC calculations based on responses to the Subsidized and Transitional Employment Demonstration 
12 -month youth survey. 

NOTES: Results in this table are regression-adjusted, controlling for pre -random assignment characteristics. 
Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
a This measure includes developing a resume, filling out job applications, preparing for job interviews, and job 
readiness training. 
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Box 4.1 


How to Read the Impact Tables in This Report 

The impact tables in this report use the same format as does the table excerpt below. In this 
case, employment and earnings outcomes are shown for the program and the control groups. 
For example, the table exceipt shows that the program group earned approximately $6,685 
over the 12-month follow-up period, while the control group earned about $3,252. 

The “Difference” column in the table exceipt shows the differences between the two research 
groups’ earnings — that is, the program’s estimated effect, or impact, on earnings. For exam- 
ple, the estimated impact on earnings in Year 1 can be calculated by subtracting $3,252 from 
$6,685, yielding a $3,433 difference. 

Differences marked with asterisks are “statistically significant,” meaning that they are larger 
than would generally be expected if the program had no true effect; that is, they are likely 
attributable to the offer of YA1P services. The number of asterisks indicates whether the 
estimated impact is statistically significant at the 10 percent (one asterisk), 5 percent (two 
asterisks), or 1 percent (three asterisks) level. The lower the level (or the more asterisks), the 
less likely that an ineffective program could have generated the impact. For example, as shown 
in the second row of data, YAIP had a statistically significant impact of $3,433 on earnings 
during Year 1; that is, youth who were offered YAIP services earned $3,433 more, on average, 
than did youth who were not offered YAIP services. This impact is statistically significant at 
the 1 percent level, which means that there is less than a 1 percent probability that an ineffec- 
tive program would have resulted in an estimated impact this large. 

In 90 percent of experiments of this type, the true value of the impact would fall within the 
range shown in the “confidence interval” column. For example, in the second example row of 
data, there is a 90 percent chance that the true value of this impact is between $3,112 and 
$3,754. 


One- Year Impacts on Employment and Earnings 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

90 Percent 

Confidence 

Interval 

Employment (%) 

95.1 

66.0 

29 o *** 

[26.9, 31.3] 

Total earnings ($) 

6,685 

3,252 

3,433 *** 

[3112, 3754] 
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services in the year following random assignment. These modest impacts also indicate that 
YAIP’s paid internship component likely produces the most substantial difference in the 
services program and control group members receive. 1 


Impacts on Employment and Earnings 

Table 4.2 presents the employment and earnings outcomes for the program and control groups 
in the first year after random assignment. 2 The top panel (administrative employment outcomes) 
shows measures derived from unemployment insurance wage records, supplemented with data 
from DYCD on subsidized employment and earnings. Figure 4.1 depicts employment and 
earnings trends from these data sources. The bottom panel of Table 4.2 (self-reported employ- 
ment outcomes) shows measures derived from the 12-month survey. 

• In the year following random assignment, program group members 
were more likely to be employed and worked in more quarters than con- 
trol group members. However, program impacts on employment dissi- 
pated by the beginning of the second year after random assignment. 

About two-thirds (66 percent) of the control group was employed in a job covered by 
unemployment insurance at some point in the year following random assignment, but nearly all 
(95 percent) of the program group was employed in this period, resulting in a program impact 
on employment of 29 percentage points. Self-reported employment in the year following 
random assignment from the 12-month survey indicates a smaller, but still positive, program 
impact on employment (7 percentage points). 3 

'Appendix Table C.l shows that only 8 percent of control group members reported participating in a paid 
internship in the four months following random assignment, compared with 60 percent of program group 
members, based on responses to the four-month survey. 

2 The YAIP internship measure shown in Table 4.2 varies from the participation measures shown in Table 
3.2 due to differences in the data source and the construction of the measure. In Table 3.2, data on participation 
in a YAIP internship is drawn from program participation data and reflects actual work in an internship. In 
Table 4.2, the measure is based on YAIP payroll data, and includes all program group members who partici- 
pated in some paid element of the program (including orientation) even if they did not go on to work in a YAIP 
internship. 

3 The difference between administrative and self-reported employment rates for the control group is likely 
a reflection of the New York City labor market, which has a fairly large proportion of non-co vered contingent 
employment related to tourism and entertainment, informal labor market employment, or both. The difference 
between administrative data and self-reported employment for the program group is possibly due to program 
group members not perceiving their participation in the program as employment (a common measurement 
issue in subsidized employment programs). It is important to note that only 60 percent of the program group 
reported employment in a paid internship (the program’s term for its subsidized employment placements) in 
the survey that Decision Information Resources, the project’s survey firm, administered during program 
participation. (See Appendix Table C.l.) 
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Table 4.2 


One-Year Impacts on Employment and Earnings 






90 Percent 


Program Control 

Difference 

Confidence 

Outcome 

Group 

Group 

(Impact) 

Interval 

Administrative employment outcomes 





Employment (%) 

95.1 

66.0 

29 o *** 

[26.9,31.3] 

YAIP subsidized employment (%) 

80.7 

- 

- 


Number of quarters employed 

2.6 

1.6 

1 Q *** 

[0.9, 1.1] 

Employed in all quarters (%) 

28.4 

13.8 

14 5 *** 

[12.1,17.2] 

Total earnings ($) 

6,685 

3,252 

3,433 *** 

[3112,3754] 

YAIP subsidized earnings ($) 

1,704 

- 

- 


Total earnings (%) 





$6,000 or more 

41.9 

21 

20 9 *** 

[18.0,23.7] 

$10,000 or more 

19.6 

9.4 

10.3 *** 

[8.0,12.5] 

$14,000 or more 

7.7 

3.7 

q j *** 

[2.6, 5.6] 

Employment in first quarter of Y ear 2 (%) 

54.1 

52.3 

1.8 

[-1.3, 5.0] 

YAIP subsidized employment in first quarter of Year 2 (%) 

0.0 

- 

- 


Sample size (total = 2,678) 

1,638 

1,040 



Self-reported employment outcomes (based on survey data) 





Ever employed in one year (%) 

81.9 

75.2 

0 1 *H=H= 

[3.7, 9.7] 

Currently employed (%) 

53.9 

51.5 

2.5 

[-1.2, 6.1] 

Full-time employment 

32.4 

28.4 

4.0 * 

[0.6, 7.5] 

Part-time employment 

21.0 

22.3 

-1.3 

[-4.3, 1.8] 

Current hourly wage (%) 





More than $8.00 

40.1 

37.0 

3.1 

[-0.5, 6.8] 

More than $10.00 

16.5 

15.1 

1.4 

[-1.4,4. 1] 

More than $12.00 

8.8 

7.4 

1.4 

[-0.7, 3.5] 

Type of employment (%) 





Not currently employed 

46.8 

49.2 

-2.4 

[-6.1, 1.3] 

Permanent 

42.9 

37.9 

5.1 ** 

[1.4, 8.7] 

Temporary, including day labor and odd jobs 

10.3 

12.9 

-2.7 * 

[-5.0, -0.3] 


(continued) 
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Table 4.2 (continued) 



Program Control Difference 

90 Percent 

Confidence 

Outcome 

Group 

Group 

(Impact) 

Interval 

Currently working in subsidized job or employment program (%) 

2.3 

1.4 

0.8 

[-0.2,1.9] 

Among those currently employed' 1 

Average hours worked per week 

31.0 

29.9 

1.1 


Average hourly wage ($) 

10.6 

10.5 

0.0 


Among those who have not worked since random 
assignment, reason not working 13 (%) 

Unable to work due to injury, illness, disability, 
pregnancy, or childbirth 

17.2 

15.6 

1.6 


Family responsibilities 

7.4 

9.2 

-1.8 


Personal issues 

8.1 

3.8 

4.2 


Attending school 

22.8 

17.4 

5.4 


Unable to find work 

36.6 

44.8 

-8.2 


Not interested in working 

4.1 

3.8 

0.3 


Other 

3.8 

5.4 

-1.5 


Sample size (total = 2,127) 

1,327 

800 




SOURCES: MDRC calculations based on quarterly wage data from the National Directory of New Hires, YAIP 
management information system subsidized earnings records, and responses to the Subsidized and Transitional 
Employment Demonstration 12-month youth survey. 

NOTES: Results in this table are regression-adjusted, controlling for pre -random assignment characteristics. 
Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
a These measures are calculated among those employed at the time of their 12-month follow-up survey 
interviews; they are therefore considered nonexperimental and are not tested for statistical significance. 

b These measures are calculated among those who had not worked during the time between random assignment 
and their 12-month follow-up survey interviews; they are therefore considered nonexperimental and are not tested 
for statistical significance. 


The program group also worked more compared with the control group: The average 
number of quarters in which program group members were employed in the year following 
random assignment was 2.6 quarters, compared with 1.6 quarters for control group members. 
Additionally, 28 percent of the program group was employed in all four quarters following 
random assignment, compared with 14 percent of the control group. 
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Average earnings ($) Percentage employed 


Figure 4.1 

Employment and Earnings Over Time 





Quarter after random assignment 
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Quarter after random assignment 

■ Program group Control group 


(continued) 
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Figure 4.1 (continued) 


SOURCES: MDRC calculations based on quarterly wage data from the National Directory of New Hires and 
YAIP management information system subsidized earnings records. 


NOTES: Results in this table are regression-adjusted, controlling for pre -random assignment characteristics. 
Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 


The relatively high levels of employment among the control group may suggest that the 
study population is not very disconnected. However, as discussed in Chapter 1, disconnection is 
not a fixed state, thus cycling in and out of school or work is common. A measure that summa- 
rizes across time, such as ever employed since random assignment, may mask this cycling. 
Consideration of the employment rate in the context of the time employed (average number of 
quarters employed) suggests that most employment in which the control group engaged in the 
year following random assigmnent was of relatively short duration. 

As Figure 4.1 depicts, the greatest difference between program and control group mem- 
bers’ employment rates measured with administrative data occurred in the first two quarters 
after random assignment. Once participation in the paid internship phase of YAIP ended, the 
employment rates of the two groups converged. By the first quarter of the second year after 
random assigmnent, the employment rates of the program and control groups were almost the 
same. 


• Program group members had higher earnings in the year following 
random assignment than did control group members; program impacts 
on earnings persisted after employment rates between the two research 
groups converged. 

Program group members also had higher earnings in the year following random as- 
sigmnent, on average earning $3,433 more than control group members. This difference is 
much larger than the $1,704 that program group members earned, on average, from YAIP 
internships. As Figure 4. 1 shows, the largest differences in earnings between the program and 
control groups were in the first and second quarters after random assigmnent (during the period 
when program group members were most active in YAIP internships), but there were statistical- 
ly significant differences in earnings in all four quarters of the first year after random assign- 
ment. This continuing difference in earnings may be due to differences in the nature of the jobs 
that program and control group members held. Sample members’ self-reported employment at 
the time of their 12-month survey interviews indicate that program group members were more 
likely to be working full-time hours in permanent jobs, compared with control group members. 
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These findings suggest that program group members may have obtained slightly higher quality 
employment compared with their control group counterparts. 

As an initial response to the question of whether the program was more effective for 
some groups compared with others — or “differential impacts” — the research team tested a 
few critical employment and earnings outcomes for a number of subgroups including study 
cohort, educational status at random assignment (participants with a high school diploma or 
equivalency certificate versus those without), length of disconnection at random assignment 
(time since last employed, in school, or in a high school equivalency program), and gender. The 
research team chose these subgroups because they are possible moderators of program impacts. 
In general, the team found significant differences in program impacts on employment rates for 
all of the subgroups — the program impacts on employment were larger for male and less 
educated sample members, members of cohorts 1 and 2, and members who had been discon- 
nected longer (see Appendix Tables D.l, D.2, D.3, and D.4). However, no differences were 
found in program impacts on earnings. Further analysis of differential impacts will be conduct- 
ed once additional follow-up is available. 4 


Impacts on Education and Training 

Table 4.3 shows education and training outcomes for the program and control groups in the first 
year after random assigmnent. The top panel of the table presents measures derived from the 
12-month survey, while the bottom panel shows measures calculated using administrative data 
from the National Student Clearinghouse. 

• Program and control group members were fairly similar in terms of 
their participation in education and training in the year after random 
assignment. 

About a third of both program group members (36 percent) and control group members 
(34 percent) reported participation in education and training in the year following random 
assigmnent based on 12-month survey data. These figures include participation in General 
Educational Development (GED) or high school diploma classes, English as a Second Lan- 
guage (ESL) classes, Adult Basic Education (ABE) classes, and vocational training. Rates of 


4 These analyses will also incorporate another source of potential variation in program impacts occurring at 
the provider location level. Appendix E shows preliminary analysis of variation by YAIP provider location. 
While these preliminary analyses are at the provider location level, the research team will explore impact 
variation among groups of providers, with the provider grouping determined by shared methods of program 
operation, target population, and other characteristics. 
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Table 4.3 


One-Year Impacts on Education and Training 






90 Percent 


Program Control Difference 

Confidence 

Outcome (%) 

Group 

Group 

(Impact) 

Interval 

Self-reported education outcomes (based on survey data) 





Participated in education and training 

36.4 

33.5 

2.9 

[-0.6, 6.3] 

GED or high school diploma classes 3 

14.2 

15.0 

-0.9 

[-3.2, 1.5] 

ESL classes' 3 

2.2 

2.0 

0.2 

[-0.9,1.3] 

ABE classes 3 

15.0 

10.0 

5.0 

*** [2.5, 7.5] 

Vocational training 

18.5 

17.7 

0.9 

[-2.0, 3.7] 

Earned a high school diploma or equivalency certificate 

10.6 

12.8 

-2.2 

[-4.5, 0.1] 

Earned professional license or certification 

13.2 

12.4 

0.8 

[-1.73.2] 

Sample size (total = 2,127) 

1,327 

800 



Administrative education outcomes 





Enrolled in a postsecondary institution 

13.0 

14.0 

-1.0 

[-3. 1,1.1] 

Enrolled in four-year college 

4.6 

4.5 

0.0 

[-13,1.4] 

Enrolled in two-year college 

8.4 

9.7 

-1.2 

[-3. 1,0.6] 

Sample size (total = 2,678) 

1,638 

1,040 




SOURCES: MDRC calculations based on responses to the Subsidized and Transitional Employment 
Demonstration 12-month youth survey and postsecondary education data from the National Student Clearinghouse. 

NOTES: Results in this table are regression-adjusted, controlling for pre-random assignment characteristics. 
Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
a GED = General Educational Development. 
b ESL = English as a Second Language. 

C ABE = Adult Basic Education. 


participation in these specific types of education and training programs were modest, but 
program group members were more likely to report participation in ABE programs compared 
with control group members. Administrative data measuring enrollment in postsecondary 
education also indicate similar levels of participation in both two- and four-year colleges across 
the two research groups. Finally, program and control group members reported similar levels of 
receipt of high school diplomas or equivalency certificates and professional licenses or certifica- 
tions. 
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As with the employment and earnings outcomes, the research team tested a few select 
education outcomes for a number of subgroups to investigate differential impacts. The team 
found no notable differences in impacts. (See Appendix Tables D.l, D.2, D.3, and D.4.) 


Impacts on Other Outcomes 

Increased engagement in productive activities can positively affect other areas of a person’s life. 
For example, job loss and unemployment have a well-documented negative impact on well- 
being, both economic and personal, while income increases are associated with an uptick in 
positive assessments of life satisfaction. Thus, participation in YAIP and any resulting increases 
in engagement in productive activity may lead to positive effects on measures of well-being. 
Likewise, engagement in productive activity may also discourage criminal activity. Tables 4.4, 
4.5, and 4.6 show measures of program and control group outcomes related to these areas 
(psychosocial outcomes, economic and personal well-being, and criminal involvement). All 
outcomes in these tables are based on the 12-month survey. 

• Program and control group members reported sim il ar levels of well- 
being at one year after random assignment. 

The research team assessed a variety of different measures, and, overall, program and 
control group members scored similarly on nearly all of them. The team also evaluated related 
indicators of personal well-being for the second cohort at roughly four months after random 
assignment, a point in time when many program group members in this cohort were participat- 
ing in YAIP. (See Appendix Table C.l.) While the program and control group outcomes were 
similar on most of the assessed psychosocial and personal well-being measures, there were a 
couple of statistically significant differences between the two research groups. Specifically, in 
the shorter term, YAIP appears to have had some positive effects on young people’s happiness 
and perceptions of obstacles to achieving their goals. 5 


Conclusion 

The interim impact results reported in this chapter indicate that YAIP was successful in increas- 
ing young people’s access to employment services and developing or strengthening supportive 
relationships for them. Program group members were more likely to have been employed and 


5 Future analyses will further consider these and additional topics, including future expectations. Appendix 
F shows preliminary analysis of early data on these topics. 


69 



Table 4.4 


One-Year Impacts on Psychosocial Outcomes 


Outcome 

Program Control Difference 
Group Group (Impact) 

90 Percent 

Confidence 

Interval 

Score on self-esteem scale 3 

3.29 

3.25 

0.04 ** 

[0.01,0.07] 

Score on career orientation scale* 3 

3.32 

3.29 

0.03 

[0.00,0.06] 

Score on social support scale 13 

3.83 

3.81 

0.02 

[-0.05,0.10] 

Score on optimism scale 3 * 

2.97 

2.95 

0.01 

[-0.02,0.05] 

Believed that one or more circumstances 
made it difficult to achieve goals 6 (%) 

42.1 

41.6 

0.6 

[-2. 9, 4.0] 

Sample size (total = 2,127) 

1,327 

800 




SOURCE: MDRC calculations based on responses to the Subsidized and Transitional Employment 
Demonstration 12-month youth survey. 

NOTES: Results in this table are regression-adjusted, controlling for pre -random assignment characteristics. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a The Rosenberg Self-Esteem Scale is a 10-item scale that assesses feelings of self-esteem. Response categories 
range from 1 = "strongly disagree" to 4 = "strongly agree," where higher scores indicate higher levels of self- 
esteem. The 10 items are averaged. 

b The career orientation scale is a six-item scale that assesses career expectations and aspirations. Scale scores 
range from 1 to 4, where higher scores indicate higher levels of career orientation. The six items are averaged. 

c The Medical Outcomes Study Social Support Survey is a seven-item scale that assesses the types of social 
support available to respondents. Scale scores range from 1 to 5, where higher scores indicate higher levels of 
social support. The seven items are averaged. 

d The optimism scale is a six-item scale that assesses feelings of optimism. Scale scores range from 1 to 4, 
where higher scores indicate higher levels of optimism. The six items are averaged. 

“Circumstances include lack of a high school diploma or equivalency certificate, limited reading or math skills, 
limited work history, unstable housing, experience in foster care, pregnancy or child care, physical or medical 
disability, mental health difficulties, criminal record, alcohol or substance abuse, and family responsibilities. 


worked more consistently than control group members in the year following random assign- 
ment, but quarterly employment rates converged after the end of the paid internships offered by 
YAIP. Program group members also had higher earnings than control group members. Earnings 
impacts were observed in every quarter of the year following random assignment, although the 
size of the impacts decreased over time. These earnings differences, along with data concerning 
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Table 4.5 


One-Year Impacts on Economic and Personal Well-Being 


Outcome (%) 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

90 Percent 

Confidence 

Interval 

Economic well-being outcomes 

Experienced financial shortfall in past 12 months 

28.5 

25.9 

2.6 

[-0.6, 5. 8] 

Could not pay rent or mortgage 

16.4 

14.9 

1.5 

[-1.1, 4.2] 

Evicted from home or apartment 

3.9 

3.2 

0.7 

[-0.7, 2.1] 

Utility or phone service disconnected 

17.2 

15.1 

2.1 

[-0.6, 4.8] 

Could not afford prescription medicine 

7.2 

7.6 

-0.5 

[-2.4, 1.4] 

Experienced food insufficiency in prior month 

15.6 

14.8 

0.8 

[-1.9, 3.5] 

Homeless or living in a shelter in prior month 

2.6 

3.1 

-0.5 

[-1.7, 0.7] 

Had health insurance coverage in prior month 

65.6 

66.6 

-1.0 

[-4.4, 2.5] 

Personal well-being outcomes 

Is currently in good, very good, or excellent health 

88.4 

88.6 

-0.2 

[-2.6, 2.2] 

Experienced serious psychological distress in past month 3 

8.0 

7.6 

0.5 

[-1.5, 2.4] 

Overall happiness 

Very happy 

24.1 

23.0 

1.1 

[-2.1, 4.2] 

Pretty happy 

59.7 

60.5 

-0.9 

[-4.5, 2.8] 

Not too happy 

16.2 

16.4 

-0.2 

[-3.0, 2. 5] 

Sample size (total = 2,127) 

1,327 

800 




SOURCE: MDRC calculations based on responses to the Subsidized and Transitional Employment Demonstration 
12-month youth survey. 

NOTES: Results in this table are regression-adjusted, controlling for pre -random assignment characteristics. 
Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
a A score of 13 or higher on the Kessler-6 Scale (K-6) is used here to define serious psychological distress. The 
K-6 assesses how often during the past month a respondent felt so sad that nothing could cheer the individual lip, 
nervous, restless or fidgety, hopeless, that everything was an effort, and worthless. As a result of minor differences 
between the scale used to administer the K-6 in the Subsidized and Transitional Employment Demonstration 12- 
month youth survey and the standard K-6, the percentages presented in this table may slightly underestimate the 
incidence of serious psychological distress among the YAIP sample. 
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Table 4.6 


One-Year Impacts on Criminal Involvement 


Outcome (%) 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

90 Percent 

Confidence 

Interval 

Arrested 

10.4 

10.5 

-0.1 

[-2.3, 2.1] 

Convicted of a crime 

2.6 

2.6 

0.1 

[-1.1, 1.2] 

Incarcerated 

3.0 

3.1 

0.0 

[-1.3, 1.2] 

On parole or probation 

3.9 

2.8 

1.1 

[-0.2, 2.4] 

Sample size (total = 2,127) 

1,327 

800 




SOURCE: MDRC calculations based on responses to the Subsidized and Transitional Employment Demonstration 
12-month youth survey. 


NOTES: Results in this table are regression-adjusted, controlling for pre -random assignment characteristics. 
Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 


current employment, suggest that program group members may have been able to secure 
somewhat better jobs (permanent positions, with full-time hours) compared with control group 
members. The research team will examine this possible program impact more fully later, when 
additional follow-up data are available. 

Aside from employment and earnings differences, program and control group members 
demonstrated similar outcomes. Their participation in education and training was comparable, 
and most indicators of well-being were roughly the same for both research groups. Again, 
further analysis of these outcomes will be reported once additional follow-up is available. 
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Chapter 5 

Conclusion and Next Steps 


This report presents implementation and early impact findings from a rigorous random assign- 
ment study of the Young Adult Internship Program (YAIP), one of several programs being 
evaluated as part of the Subsidized and Transitional Employment Demonstration (STED). 
While a future report will discuss the impacts of YAIP 30 months after study enrollment, these 
early findings provide important insights into the implementation of a relatively “light-touch” 
intervention targeted to a higher functioning, but still disadvantaged, group of disconnected 
youth in New York City. This chapter highlights key findings from the implementation study of 
YAIP and summarizes the early (one-year) impacts of the program. The chapter concludes by 
noting next steps in the YAIP evaluation. 


Key Implementation Findings 

The research team assessed YAIP’s implementation using several different data sources, 
including (but not limited to) interviews with key provider staff as well as YAIP management 
staff from DYCD and CEO, internship worksite observations, analysis of data collected from all 
sample members when they enrolled in the study, and participation data from the YAIP man- 
agement information system. Overall, YAIP was implemented as expected; nevertheless, 
program implementation was not without its challenges. Key findings from the implementation 
study are as follows: 

• YAIP providers implemented a comprehensive, multistage screening 
and intake process to identify their target population — New York 
City’s most job-ready disconnected youth — for enrollment into the 
study. Compared with local and national populations of disconnected youth, 

YAIP sample members were better positioned on various socioeconomic in- 
dicators. They were more likely to have a high school credential and previous 
work experience and less likely to receive public benefits or to live in unsta- 
ble housing situations. Thus, while the YAIP sample certainly still faced a 
number of serious challenges (for instance, the median length of time since 
study sample members were last in school, enrolled in a GED program, or 
working was nine months), providers appear to have been successful in iden- 
tifying and enrolling the more advantaged subset of disconnected youth that 
they believed to be best able to benefit from the program. 
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• While there was some variation between YAIP providers in terms of 
staff roles, agency resources, and the overall mission of each organiza- 
tion, providers delivered the YAIP model consistently and adhered very 
closely to the model as designed. All YAIP providers delivered a three- 
phase program which included a 2- to 4- week orientation, a 10- to 12-week 
paid internship with mandatory weekly educational workshops, and a nine- 
month follow-up period during which case management and follow-up ser- 
vices were offered to participants. Young people were paid the minimum 
wage for orientation, internship placements, and weekly educational work- 
shops. 

• Overall, participation in YAIP was high: Over three-fourths of young 
people assigned to the program group participated in a subsidized in- 
ternship and, among this group, approximately 86 percent completed 
their internships. The average length of program participation among young 
people who worked in a subsidized internship was 1 5 weeks (this figure in- 
cludes time between random assigmnent and the orientation phase), with an 
average of 22 hours per week spent working in internships and attending ed- 
ucational workshops during the internship phase of the program. 

• Provider staff faced many challenges in recruiting participants and de- 
livering the program, leaving them vulnerable to burnout. Program man- 
agers reported difficulties with frequent staff turnover as a result. Staff chal- 
lenges included navigating a lengthy and involved recruitment and 
assessment process; contending with YAIP’s overlapping program cycles, 
which required staff to juggle varying responsibilities for cohorts in different 
phases of the program simultaneously; the inherent difficulties of working 
with a disadvantaged group of young people, some of whom require a great 
deal of support even outside of regular working hours; and handling DYCD’s 
oversight and perfonnance expectations, which many staff viewed as restric- 
tive and unrealistic. 

• YAIP provider staff, supervisors at internship sites, and participants 
seemed to agree that while the internship is a positive learning experi- 
ence for participants, it may not sufficiently enhance their long-term 
employability. YAIP was originally conceived of as a relatively “light- 
touch” intervention to help reengage a more job-ready subset of the discon- 
nected youth population based on the premise that this group would not re- 
quire intensive services to get back on track. However, many of those in- 
volved with the YAIP program disagreed with this premise and believed that 


74 



youth required both a higher level of support and a longer intervention to im- 
prove their educational and labor market outcomes. 

Based on the results of the implementation study, the program effectively “reconnect- 
ed” many of the disconnected youth they served during the study period by placing them in paid 
internships. The next section presents results from an early analysis of whether reengagement 
improved young people’s outcomes in employment and earnings, education and training, and 
other areas of interest. 


Early Impacts of YAIP 

The impact study relies on data from three key sources: employment and earnings data from the 
National Directory of New Hires (NDNH), 12-month survey data, and postsecondary school 
enrollment data from the National Student Clearinghouse. For this report, follow-up of just over 
one year after random assignment was available to assess differences in outcomes between 
program and control group members. Early findings from the impact study are as follows: 

• There was a statistically significant service differential between program 
and control group members wherein the program group was more like- 
ly to have received help finding and keeping a job than the control 
group; however, many control group members also received help in this 
area. Program group members were also more likely than control group 
members to have received advice or support and mentorship from staff 
members at an agency or organization. 

• Program group members were more likely than control group members 
to have worked at some point in the year following random assignment, 
but the quarterly employment rates of the two research groups con- 
verged shortly after the subsidized internships provided by YAIP ended. 

Program group members also had higher earnings than control group mem- 
bers in the year following random assigmnent; while highest during the time 
when program group youth were working in paid internships, earnings im- 
pacts persisted throughout the follow-up period. Sustained earnings differ- 
ences toward the end of the follow-up period combined with information 
from the 12-month survey concerning current employment suggest that pro- 
gram group youth may have been working in better jobs than control group 
members (that is, permanent jobs with full-time hours) at the end of the fol- 
low-up period. 
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• Program and control group members had similar outcomes over the 
first year of follow-up in other domains where impacts were assessed. 

These domains include education and training, psychosocial outcomes, eco- 
nomic and personal well-being, and criminal involvement. 

The impacts of YAIP will be more fully examined later, when data from 30-months of 
follow-up are available. 


Next Steps 

The STED evaluation of YAIP is part of a larger effort to understand how best to help young 
people who have become untethered from the worlds of school and work to reengage in 
productive activity. Findings from the implementation study indicate that YAIP is a well- 
implemented program, operates similarly across providers with a high degree of fidelity to the 
intended program model, and serves a large swath of New York City’s more job-ready discon- 
nected youth. Rates of participation are high, a notable finding considering the many youth 
programs that struggle to keep young people engaged in their services. Whether YAIP is having 
its ultimate intended effects of improving participants’ labor market prospects and reducing 
their risk of long-term economic hardship remains an open question. The current report presents 
only one-year impacts of the program. It is too early to draw any firm conclusions about 
whether YAIP will improve employment outcomes, or outcomes in other domains, in the longer 
term. Final impact results, with a longer-term follow-up period of 30 months, will be presented 
in a later report, as will the results of a benefit-cost study. 


76 



Appendix A 


Survey Response Bias Analysis 



THIS PAGE INTENTIONALLY LEFT BLANK 



The YAIP evaluation includes three surveys. This analysis examines the survey response for the 
first two surveys, one administered at roughly four months post-random assignment (4-month 
survey) and the second administered roughly one year post-random assignment (12-month 
survey). 1 A subset of the full research sample completed each survey. Therefore, it is possible 
that those who participated in the surveys are not representative of the full research sample, 
which could introduce bias into the estimates produced from the survey data. It is not unex- 
pected that each of the survey samples may differ slightly from the full research sample in terms 
of socio-demographic characteristics, since certain characteristics such as age, gender, and 
stability are generally associated with survey response. The main concern is differences be- 
tween program and control group respondents — if there are differences between the type of 
program group members who responded to each survey and the type of control group members 
who responded to each survey, impact estimates based on the survey data may be biased. 

Overall, the administration of both surveys went well. Both surveys achieved response 
rates of about 80 percent and nearly all interviews were completed on time (within the survey 
fielding window of four months). However, the 4-month survey was administered only to the 
second cohort of the YAIP study sample due to budget and logistical constraints. As a result, the 
respondent sample for that survey (n=719) is smaller than the respondent sample for the 12- 
month survey (n=2,127). 

Despite the differences in fielded samples, the response analysis for the two surveys in- 
dicates limited evidence of issues that might introduce bias into impact estimates. There are a 
few small differences in the socio-demographic characteristics of the survey respondents 
compared with nonrespondents for both surveys, which is a typical finding of survey response 
analysis as certain characteristics are associated with survey response propensity. However, the 
baseline characteristics of the members of the two research groups are similar within the 
samples for both surveys. In addition, comparisons of program impacts among survey respond- 
ents with those estimated for the full research sample using administrative data show that while 
the magnitude of impacts varies slightly across the samples, the overall pattern of results is the 
same. 


Response Differences 

To test whether survey respondents differ from nonrespondents, the research team compared the 
socioeconomic characteristics of these two groups. As shown in Appendix Table A.1, the 


'Results from the third survey, administered at roughly 30 months after random assignment, will be pre- 
sented in a later report, as will the response bias analysis results for that survey. 
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Appendix Table A.l 


Selected Baseline Characteristics of Survey Respondents and Nonrespondents, 

by Survey Wave 

4-Month Survey 12-Month Survey 

Non- Non- 

Characteristic Respondents respondents Respondents respondents 


Average age (years) 

20.5 

20.5 

20.6 

20.8 

Female (%) 

50.7 

44.2 

51.9 

47.2 ** 

Race/ethnicity (%) 





Hispanic 

38.0 

40.9 

35.3 

39.5 

Black, non-Hispanic 

58.8 

55.2 

59.1 

54.3 

Other, non-Hispanic 

3.2 

3.9 

5.6 

6.2 

Ever employed (%) 

70.0 

66.9 

72.1 

69.9 

Worked in last three months (%) 

25.8 

26.0 

29.7 

31.5 

Has children (%) 

18.3 

21.7 

19.9 

20.1 

Has high school diploma or GED certificate (%) 

63.5 

g *** 

63.3 

55.5 *** 

Serious barrier to employment 3 (%) 

42.1 

51.4 ** 

44.3 

2 *** 

Receives public assistance (%) 

25.9 

29.3 

25.8 

28.1 

Ever arrested (%) 

25.9 

32.2 * 

24.8 

31.0 *** 

Sample size 

719 

181 

2,127 

551 


SOURCE: MDRC calculations based on data from MDRC's random assignment system and the YAIP 
management information system. 

NOTE: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
The 4-month survey was fielded to members of the second cohort only. 

a This measure includes homelessness or unstable housing, history of foster care, limited literacy or math 
skills, mental or physical disability, previous criminal conviction, and pregnancy or having a child. 


respondents to both the 4-month and 12-month follow-up surveys had statistically significant 
differences from nonrespondents for a few characteristics. For both surveys, respondents were 
more likely to have a high school diploma or GED certificate, less likely to have a serious 
barrier to employment, and less likely to have ever been arrested. Additionally, respondents to 
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the 12-month survey were more likely to be female. Since comparison of a series of characteris- 
tics causes susceptibility to false positives, a global test of the relationship of these characteris- 
tics to response status was conducted. 2 This test is conducted by estimating a regression model 
predicting survey response; the test statistic reported for each characteristic indicates whether 
that characteristic has a statistically significant association with survey response controlling for 
the other characteristics, and the joint test indicates whether the characteristics collectively have 
a statistically significant association with survey response. Appendix Table A.2 presents the 
results of these tests for both surveys. For the 4-month survey, a few characteristics have 
significant effects, meaning that the characteristic, when controlling for the other characteristics 
in the model, has a significant association with survey response status; the overall joint test is 
not statistically significant, indicating that response status for this survey cannot be predicted 
using these characteristics. Several characteristics in the joint test for the 12-month survey 
response are also significant, and the overall test is statistically significant as well. 

It is not uncommon to find baseline characteristics that are predictive of response status. 
These associations may indicate some level of nonresponse bias, but this bias would primarily 
affect level estimates rather than impact estimates, since the bias affects both program and 
control group members. (Generally, survey respondents tend to be more stable than non- 
respondents. Thus, outcome levels for respondents may overstate positive overall outcome 
levels to some degree.) In regard to the estimation of program impacts, differences between 
respondents by research group are the primary concern. Accordingly, the research team com- 
pared socio-demographic characteristics of survey respondents by research group. As shown in 
Appendix Table A.3, survey respondents were fairly similar across research groups. While the 
joint test of the association between socio-demographic characteristics and research groups for 
survey respondents was not significant in either survey wave (Appendix Table A.4), one 
characteristic, whether the respondent had a high school diploma or GED certificate at baseline, 
was significant for 12-month survey respondents. This indicates that survey respondents in the 
program group were slightly less likely to have had a high school diploma or GED certificate at 
study enrollment compared with survey respondents in the control group, controlling for the 
other characteristics. 


Impact Differences 

Another way to assess possible bias from survey response is to examine differences in impacts 
measured with administrative data between the survey respondent samples and the full research 


2 As the number of individual tests conducted increases, the likelihood of finding a statistically significant 
difference increases. 
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Appendix Table A.2 


Joint Test of Differences Between Survey Respondents and Nonrespondents, 

by Survey Wave 


Characteristic 

4-Month Survey 

12 -Month Survey 

Age 

0.615 

1.854 

Race/ethnicity 

0.957 

4.360 

Female 

2.720 * 

2.446 

Ever employed 

0.236 

1.372 

Worked in last three months 

0.110 

0.559 

Children 

0.018 

2.090 

High school diploma or GED certificate 

4.258 ** 

6.574 ** 

Serious barrier to employment 11 

1.940 

5.533 ** 

Receives public assistance 

0.386 

0.264 

Ever arrested 

0.281 

1.927 

Overall test 

16.224 

34.539 *** 

Sample size 

900 

2,678 


SOURCE: MDRC calculations based on data from MDRC's random assignment system and the YAIP 
management information system. 

NOTES: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
The 4-month survey was fielded to members of the second cohort only. 

The joint test of differences between survey respondents and nonrespondents was conducted by estimating a 
regression model of response probability for each survey wave. For each characteristic, the joint test indicates 
whether there is an association between the characteristic and response, controlling for the other characteristics. 
The overall test indicates whether there is an association between survey response and all of the characteristics 
collectively. 

a This measure includes homelessness or unstable housing, history of foster care, limited literacy or math skills, 
mental or physical disability, previous criminal conviction, and pregnancy or having a child. 
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Appendix Table A.3 


Selected Baseline Characteristics of Survey Respondents, 
by Survey Wave and Research Group 


Characteristic 

4-Month Survey 

Program Control 
Group Group 

12-Month Survey 

Program Control 

Group Group 

Average age (years) 

20.5 

20.5 

20.6 

20.7 

Female (%) 

49.5 

52.6 

52.2 

51.3 

Race/ethnicity (%) 





Flispanic 

37.0 

39.6 

34.9 

36.1 

Black, non-Hispanic 

58.9 

58.6 

59.5 

58.4 

Other, non-Hispanic 

4.1 

1.8 

5.6 

5.5 

Ever employed (%) 

71.4 

67.8 

71.7 

73.0 

Worked in last three months (%) 

24.3 

28.1 

29.5 

30.1 

Has children (%) 

18.5 

18.1 

19.6 

20.5 

Has high school diploma or GED certificate (%) 

62.3 

65.6 

61.8 

65.7 * 

Serious barrier to employment' 1 (%) 

41.4 

43.2 

43.4 

45.8 

Receives public assistance (%) 

25.5 

26.6 

25.7 

26.0 

Ever arrested (%) 

25.2 

26.9 

23.7 

26.6 

Sample size 

443 

276 

1,327 

800 


SOURCE: MDRC calculations based on data from MDRC's random assignment system and the YAIP management 
information system. 

NOTES: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The 4-month survey was fielded to members of the second cohort only. 

a This measure includes homelessness or unstable housing, history of foster care, limited literacy or math skills, 
mental or physical disability, previous criminal conviction, and pregnancy or having a child. 


sample. If the differences between the program and control groups in the survey respondent 
sample are not similar to those observed for the full research sample, it would indicate that the 
survey respondent sample is not representative, and thus survey-based impact estimates may be 
biased. 
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Appendix Table A.4 


Joint Test of Differences Between Research Groups, 
by Survey Wave 


Characteristic 

4-Month Survey 

12-Month Survey 

Age 

0.218 

0.025 

Race/ethnicity 

2.895 

0.643 

Female 

1.032 

0.015 

Ever employed 

1.481 

0.213 

Worked in last three months 

1.654 

0.000 

Children 

0.317 

0.081 

Migh school diploma or GED certificate 

1.242 

4.478 ** 

Serious barrier to employment" 1 

0.308 

0.843 

Receives public assistance 

0.013 

0.016 

Ever arrested 

0.618 

2.683 

Overall test 

8.084 

8.814 

Sample size 

719 

2,127 


SOURCE: MDRC calculations based on data from MDRC's random assignment system and the YAIP 
management information system. 

NOTES: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
The 4-month survey was fielded to members of the second cohort only. 

The joint test of differences between research groups among survey respondents was conducted by estimating a 
regression model of response probability for each survey wave. For each characteristic, the joint test indicates 
whether there is an association between the characteristic and response, controlling for the other characteristics. 
The overall test indicates whether there is an association between survey response and all of the characteristics 
collectively. 

a This measure includes homelessness or unstable housing, history of foster care, limited literacy or math skills, 
mental or physical disability, previous criminal conviction, and pregnancy or having a child. 
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Appendix Table A.5 presents selected one-year impacts based on administrative data 
for the research and survey respondent samples. While the magnitude of the impacts varies 
slightly between samples (particularly for the 4-month survey, for which impacts are more 
positive), the overall pattern is the same, with statistically significant increases in employment 
and earnings for all three samples and no statistically significant impacts on postsecondary 
enrollment. 

A second method to assess whether impact estimates are biased due to survey nonre- 
sponse is multiple imputation. This method uses statistical modeling to predict the responses for 
sample members that did not participate in the survey. Multiple predictions are generated to 
simulate the distribution of responses from which full sample estimates are produced. In other 
words, this analysis provides an estimate of the survey-based impacts if the full research sample 
had participated in the survey. Appendix Table A.6 shows the estimated regression coefficients 
for the program effects for the 12-month survey respondent sample and the full research sample 
as estimated via multiple imputation. 3 The estimates are virtually the same for both outcomes 
among both samples, both in terms of size and statistical significance, suggesting little differ- 
ence in program impacts between survey respondents and nonrespondents. 


’Estimated coefficients are analogous to program impacts as reported in the text. 
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Appendix Table A.5 

Selected One-Year Impacts for the Research and Survey Respondent Samples 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

90 Percent 

Confidence 

Interval 

Employment (%) 

Research sample 

95.1 

66.0 

29 o *** 

[26.9,31.3] 

Respondent sample, 12-month survey 

96.0 

65.8 

30 2 *** 

[27.8,32.6] 

Respondent sample, 4-month survey 

97.9 

60.0 

38.0 *** 

[33.9,42.1] 

Total earnings ($) 

Research sample 

6,685 

3,252 

3,433 *** 

[3112,3754] 

Respondent sample, 12-month survey 

6,815 

3,225 

3,591 *** 

[3226,3956] 

Respondent sample, 4-month survey 

6,713 

2,934 

3,779 *** 

[3136,4422] 

Enrolled in postsecondary institution (%) 

Research sample 

13.0 

14.0 

-1.0 

[-3. 1,1.1] 

Respondent sample, 12-month survey 

13.9 

13.9 

0.0 

[-2.5, 2.4] 

Respondent sample, 4-month survey 

14.9 

10.9 

4.0 

[-0.1,8.!] 

Sample size 

Research sample (total = 2,678) 

Respondent sample, 12-month survey (total = 2,127) 
Respondent sample, 4-month survey (total = 719) 

1,638 

1,327 

443 

1,040 

800 

276 




SOURCES: MDRC calculations based on quarterly wage data from the National Directory of New Hires and 
postsecondary education data from the National Student Clearinghouse. 

NOTES: Results in this table are regression-adjusted, controlling for pre-random assignment characteristics. 
Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
Employment rates and earnings include both YA1P subsidized jobs and all other jobs covered by 
unemployment insurance. 
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Appendix Table A.6 


Estimated Regression Coefficients for Program Impacts 
for 12-Month Survey and Research Sample (Imputed) 


Outcome (%) 

Program Impact 

Ever employed in one year 

Respondent sample, 12-month survey 

y 2 *** 

Research sample (imputed) 

2 2 *** 

Currently employed 

Respondent sample, 12-month survey 

3.2 

Research sample (imputed) 

3.3 

Respondent sample, 12-month survey 

2,127 

Research sample (imputed) 

2,678 


SOURCE: MDRC calculations based on data from MDRC's random assignment system, the YAIP management 
information system, and responses to the Subsidized and Transitional Employment Demonstration 12-month youth 
survey. 

NOTES: Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

The multiple imputation results presented here estimate the program impacts on survey-based outcomes by 
performing multiple impuations of probable responses for survey nonrespondents. 
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YAIP Employer/Supervisor and Participant Program 
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Appendix Table B.l 

Employer and Worksite Supervisor Perspectives on YAIP 


Measure (%) 

Respondents 

Reasons for participating in YAIP a 


Help participants upgrade skills and training 

86.2 

Help stuggling community with new jobs 

48.3 

Test new workers for potential permanent hire 

40.7 

Short-term labor 

20.0 

Test a partnership for job placement services 

17.2 

Other 

4.1 

Sample size 

145 

Among those identified as internship worksite supervisors 

Believes part of role is to help participants learn soft skills 15 

81.6 

Believes part of role is to help participants learn hard skills’ 5 

53.9 

Uses YAIP staff as a resource to help resolve problems with participants’ 5 

42.5 

Preparation of participants for first day compared with typical entry-level workers 


More 

9.2 

About the same 

29.4 

Less 

61.5 

Performance improvement of participants compared with typical entry-level workers 


More 

69.7 

About the same 

22.0 

Less 

8.3 

Sample size 

115 


SOURCE: MDRC calculations from the Subsidized and Transitional Employment Demonstration employer or 
worksite supervisor questionnaire. 

NOTE: Among the full sample of 145 respondents, 115 held dual roles as both employers and direct supervisors of 
YAIP interns. The remaining 30 respondents were employers only. 

“Respondents selected their top three reasons from this list. For this reason, the percent of employers selecting 
each reason sum to more than 100 percent. 

b Respondents were saked about their level of agreement with the statement on a scale of 1 to 7, where 1 
indicates strong disagreement and 7 indicates strong agreement. The figure shown represents the percent of 
respondents who selected a 6 or 7 on the scale. 
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Appendix Table B.2 
YAIP Participant Program Experiences 


Measure (%) 

Participants 

Typical tasks completed at internship" 1 

General office work 

52.4 

Computer-based work 

49.4 

Interacting with customers 

47.8 

Cleaning or maintaining work spaces 

45.2 

Answering or making telephone calls 

42.7 

Services received from YAIP C 

Career planning or job search assistance 11 

79.0 

Help forming educational or career goals 

71.7 

Money management support 

57.9 

Help obtaining work-appropriate clothing 

46.9 

Transportation assistance 

37.4 

Decided to go back to school because of YAIP participation 

45.3 

Sample size 

570 


SOURCE: MDRC calculations from the Subsidized and Transitional Employment Demonstration youth 
participant questionnaire. 

NOTES: a Participants selected all that applied from among 16 unique task categories; the five most commonly 
selected categories are displayed here. 

b This category includes activities such as filing and making copies. 

Participants selected all that applied from among 18 unique service categories; the five most commonly 
selected categories are displayed here. 

d This category includes resume writing and job interview preparation. 
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Appendix Table C.l 


Four-Month Impacts on Employment, 
Psychosocial Outcomes, and Personal Well-Being 






90 Percent 


Program Control Difference 

Confidence 

Outcome 

Group 

Group 

(Impact) 

Interval 

Employment 





Ever employed since random assignment (%) 

71.5 

37.7 

33.8 *** 

[27.9,39.7] 

Ever employed in a paid internship (%) 

60.2 

8.1 

52 1 *** 

[46.7,57.4] 

Currently employed (%) 

50.2 

22.2 

28.0 *** 

[22.0,33.9] 

Full-time employment 

7.2 

9.8 

-2.5 

[-6. 1,1.0] 

Part-time employment 

42.7 

12.2 

30.6 *** 

[25.0,36.2] 

Among those currently employed, average hours worked per week a 

24.2 

30.0 

-5.7 


Psychosocial outcomes 





Average score on self-esteem scale b 

3.3 

3.3 

0.0 

[0.0, 0.1] 

Average score on career orientation scale 1 

3.4 

3.5 

0.0 

[-0.1, 0.0] 

Average score on social support scale 11 

3.8 

3.8 

0.0 

[-0.1, 0.1] 

Average score on locus of control scale 1- 

5.4 

5.4 

0.0 

[-0.1, 0.1] 

Believed that one or more circumstances made it difficult 





to achieve goals (%) 

49.2 

58.7 

_g ^ H=H=H= 

[-15. 5, -3.5] 

Has someone who could complete a small favor (%) 

84.2 

87.8 

-3.6 

[-8. 1,0.8] 

Has someone who could lend them $250 (%) 

68.2 

67.4 

0.8 

[-5. 1,6.6] 

Has a... (%) 





Mentor 

67.0 

63.5 

3.4 

[-2.6, 9.5] 

Role model 

47.4 

45.5 

1.9 

[-4.5, 8.3] 

Friend attending or who graduated from college 

72.6 

73.5 

-0.9 

[-6.5, 4.8] 

Friend earning more than $30,000 per year 

34.1 

37.7 

-3.6 

[-10.1,2.9] 

(continued) 
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Table C.l (continued) 


90 Percent 

Program Control Difference Confidence 
Outcome Group Group (Impact) Interval 


Personal well-being (%) 


Experienced serious psychological distress in past month 8 

6.4 

6.9 

-0.5 

[-3.7, 2.6] 

Overall happiness 

Very happy 

28.1 

17.6 

IQ 4 *** 

[4.9,15.9] 

Pretty happy 

61.1 

60.4 

0.6 

[-5. 6 , 6.9] 

Not too happy 

10.9 

21.9 

-11 1 *** 

[-15.6, -6.5] 

Sample size (total = 719) 

443 

276 




SOURCE: MDRC calculations based on responses to the Subsidized and Transitional Employment Demonstration 
4-month youth survey. 

NOTES: Results in this table are regression-adjusted, controlling for pre -random assignment characteristics. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Only members of Cohort 2 completed the four-month survey. 

a These measures are calculated among those employed at the time of their 4-month survey interviews; they are 
therefore considered nonexperimental and are not tested for statistical significance. 

b The Rosenberg Self-Esteem Scale is a 10-item scale that assesses feelings of self-esteem. Response categories 
range from 1 = "strongly disagree" to 4 = "strongly agree," where higher scores indicate higher levels of self- 
esteem. The 10 items are averaged. 

c The career orientation scale is a six-item scale that assesses career expectations and aspirations. Scale scores 
range from 1 to 4, where higher scores indicate higher levels of career orientation. The six items are averaged. 

d The Medical Outcomes Study Social Support Survey is a seven-item scale that assesses the types of social 
support available to respondents. Scale scores range from 1 to 5, where higher scores indicate higher levels of 
social support. The seven items are averaged. 

e The locus of control scale is a five-item scale that assesses the feelings of control over one's life circumstances. 
Scale scores range from 1 to 6 , where higher scores indicate higher locus of control. The five items are averaged. 

•Circumstances include lack of a high school diploma or equivalency certificate, limited reading or math skills, 
limited work history, unstable housing, experience in foster care, pregnancy or child care, physical or medical 
disability, mental health difficulties, criminal record, alcohol or substance abuse, and family responsibilities. 

g A score of 13 or higher on the Kessler -6 Scale (K- 6 ) is used here to define serious psychological distress. The 
K -6 assesses how often during the past month a respondent felt so sad that nothing could cheer the individual up, 
nervous, restless or fidgety, hopeless, that everything was an effort, and worthless. As a result of minor differences 
between the scale used to administer the K -6 in the Subsidized and Transitional Employment Demonstration 12- 
month youth survey and the standard K- 6 , the percentages presented in this table may slightly underestimate the 
incidence of serious psychological distress among the YA1P sample. 
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Appendix Table D.l 

One-Year Impacts on Primary Outcomes, by Cohort 







Difference 





90 Percent 

Between 


Program 

Control 

Difference 

Confidence 

Subgroup 

Outcome 

Group 

Group 

(Impact) 

Interval 

Impacts 3 

Cohort 1 

Education (%) 

Earned high school diploma or 






equivalency certificate (based on survey) 

11.7 

12.9 

-1.3 

[-5.5, 2.9] 


Enrolled in a postsecondary institution 

7.8 

11.8 

_4 0 ** 

[-7.2, -0.8] 

f 

Enrolled in four-year college 

3.0 

4.2 

-1.2 

[-33,0.9] 


Enrolled in two-year college 

4.7 

8.0 

-3.2 * 

[-5. 9, -0.5] 

f 

Employment and earnings 

Employment (%) 

96.0 

64.6 

31.3 *** 

[27.5,35.1] 

fff 

YAIP subsidized employment (%) 

81.8 

- 

- 



Average total earnings ($) 

6,755 

3,054 

3,701 *** 

[3110,4292] 


Average number of quarters employed 

2.7 

1.6 

j j *** 

[1.0,13] 

tft 

Average quarterly employment (%) 

68.4 

40.4 

28.1 *** 

[24.5,31.6] 

fff 

Employment in first quarter of Year 2 (%) 

56.2 

52.6 

3.6 

[-2.1,93] 


Currently employed (based on survey) (%) 

53.5 

50.4 

3.1 

[-3.4, 9.6] 


Sample size (total = 868) 

534 

334 




Cohort 2 

Education (%) 

Earned high school diploma or 






equivalency certificate (based on survey) 

11.2 

15.1 

-3.9 

[-8.1,03] 


Enrolled in a postsecondary institution 

13.9 

11.5 

2.4 

[-1.2, 6.0] 

f 

Enrolled in four-year college 

3.4 

3.1 

0.2 

[-1.8, 2.2] 


Enrolled in two-year college 

10.5 

8.3 

2.2 

[-1.1, 5.4] 

f 

Employment and earnings 

Employment (%) 

95.9 

59.8 

36.0 *** 

[32.2,39.9] 

fff 

YAIP subsidized employment (%) 

79.6 

- 

- 



Average total earnings ($) 

6,419 

3,059 

3,360 *** 

[2780,3940] 


Average number of quarters employed 

2.7 

1.4 

j 3 *** 

[1.2, 1.5] 

fff 

Average quarterly employment (%) 

68.1 

34.5 

33.6 *** 

[30.2,37.0] 

fff 

Employment in first quarter of Year 2 (%) 

53.5 

50.5 

3.0 

[-2.4, 8.5] 


Currently employed (based on survey) (%) 

57.0 

50.2 

6.7 * 

[0.4,13.0] 


Sample size (total = 900) 

551 

349 





(continued) 
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Appendix Table D.l (continued) 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

90 Percent 

Confidence 

Interval 

Difference 

Between 

Subgroup 

Impacts 3 

Cohort 3 

Education (%) 

Earned high school diploma or 

equivalency certificate (based on survey) 

9.2 

10.6 

-1.5 

[-5.2,2.3] 


Enrolled in a postsecondary institution 

17.1 

18.4 

-1.3 

[-5.4,2.8] 

t 

Enrolled in four-year college 

7.3 

5.9 

1.4 

[-1.4,4.1] 


Enrolled in two-year college 

9.8 

12.7 

-2.9 

[-6.4,0.61 

t 

Emnlovment and earnings 

Employment (%) 

93.8 

73.2 

20.5 *** 

[16.9,24.2] 

fff 

YA1P subsidized employment (%) 

Average total earnings ($) 

80.6 

6,739 

3,847 

2,892 *** 

[2416,3368] 


Average number of quarters employed 

2.2 

1.8 

0.5 *** 

[0.3,0.6] 

fff 

Average quarterly employment (%) 

56.0 

44.2 

j j g *** 

[8.6,14.9] 

fff 

Employment in first quarter of Year 2 (%) 

52.8 

53.8 

-1.0 

[-6.4,4.4] 


Currently employed (based on survey) (%) 

51.4 

53.5 

-2.1 

[-8.4,4.2] 


Sample size (total = 910) 

553 

357 





SOURCES: MDRC calculations based on quarterly wage data from the National Directory of New Hires, 
postsecondary education data from the National Student Clearinghouse, and responses to the Subsidized and 
Transitional Employment Demonstration 12-month youth survey. 

NOTES: Program impacts were calculated separately for each subgroup, using an ordinary least squares model 
and adjusting for pre -random assignment characteristics. Impact estimates were then examined for statistically 
significant differences across subgroups. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
a When comparing impacts between two subgroups, an H-statistic is generated. The H-statistic is used to assess 
whether the difference in impacts between the subgroups is statistically significant. Statistically significant 
differences between subgroups are indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 
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Appendix Table D.2 

One- Year Impacts on Primary Outcomes, by Educational Status at Baseline 



Has High School Diploma or GED Certificate 

Does Not Have 

High School Diploma or GED Certificate 

Difference 

Outcome 

Program Control 

Group Group 

Difference 

(Impact) 

90 Percent 

Confidence 

Interval 

Program Control 

Group Group 

Difference 

(Impact) 

90 Percent 

Confidence 

Interval 

Between 

Subgroup 

Impacts 3 

Education (%) 

Earned high school diploma or 

equivalency certificate (based on survey) 

8.6 

11.0 

-2.4 

[-5.0,0.3] 

14.2 

15.1 

-0.9 

[-5.2,33] 


Enrolled in a postsecondary institution 

18.8 

20.4 

- 1.6 

[-4.7, 1.6] 

3.9 

4.4 

-0.5 

[-2.5, 1.6] 


Enrolled in four-year college 

6.6 

6.1 

0.5 

[-1.5, 2.4] 

0.9 

1.8 

-0.9 

[-2.0,03] 


Enrolled in two-year college 

12.2 

14.5 

-2.4 

[-5. 1,0.4] 

3.0 

2.6 

0.4 

[-1.4, 2.2] 


Employment and earnings 

Employment (%) 

97.0 

71.4 

25.6 *** 

[23.0,28.2] 

92.0 

57.0 

35 Y *** 

[31.2,39.0] 

ftt 

YAIP subsidized employment (%) 

Average total earnings ($) 

82.4 

7,406 

3,777 

3,629 *** 

[3242,4017] 

77.7 

5,569 

2,428 

3,141 *** 

[2591,3692] 


Average number of quarters employed 

2.7 

1.8 

j 0 *** 

[0.9, 1.1] 

2.3 

1.3 

Y Q *** 

[0.9, 1.2] 


Average quarterly employment (%) 

68.0 

44.1 

23 g *** 

[21.4,26.3] 

58.1 

32.4 

25 y *** 

[22.4,28.9] 


Employment in first quarter of Y ear 2 (%) 

60.3 

56.2 

4.1 * 

[ 0 . 1 , 8 . 1 ] 

44.6 

46.2 

-1.7 

[-6.9,3.6] 


Currently employed (based on survey) (%) 

58.8 

54.5 

4.3 

[- 0 . 2 , 8 . 9] 

46.2 

47.9 

-1.7 

[-7.9, 4.5] 


Sample size (total = 2,666) 

988 

656 



644 

378 





SOURCES: MDRC calculations based on quarterly qage data from the National Directory of New Hires, postsecondary education data from the National 
Student Clearinghouse, and responses to the Subsidized and Transitional Employment Demonstration 12-month youth survey. 


NOTES: Program impacts were calculated separately for each subgroup, using an ordinary least squares model and adjusting for pre -random assignment 
characteristics. Impact estimates were then examined for statistically significant differences across subgroups. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

“When comparing impacts between two subgroups, an H-statistic is generated. The H-statistic is used to assess whether the difference in impacts between the 
subgroups is statistically significant. Statistically significant differences between subgroups are indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 



Appendix Table D.3 


One-Year Impacts on Primary Outcomes, by Length of Time 
Disconnected from Work and School at Baseline 




Nine Months or More 



Less Than Nine Months 


Difference 

Outcome 

Program Control 

Group Group 

Difference 

(Impact) 

90 Percent 

Confidence 

Interval 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

90 Percent 

Confidence 

Interval 

Between 

Subgroup 

Impacts 3 

Education (%) 

Earned high school diploma or 

equivalency certificate (based on survey) 

10.7 

11.8 

- 1.0 

[-4.3,2.3] 

10.1 

14.9 

_4 g ** 

[-8.2, -1.4] 


Enrolled in a postsecondary institution 

10.6 

12.6 

- 2.0 

[-4.9,0.8] 

15.4 

14.8 

0.6 

[-2.5,3.8] 


Enrolled in four-year college 

3.8 

3.7 

0.1 

[-1.7,1.8] 

5.2 

5.3 

- 0.1 

[-2.U.9] 


Enrolled in two-year college 

6.8 

8.9 

- 2.1 

[-4.5,0.41 

10.2 

9.9 

0.2 

[-2.5,3.0] 


Employment and earnings 

Employment (%) 

94.5 

58.8 

35 7 *** 

[32.4,38.9] 

95.9 

73.2 

22 7 *** 

[19.8,25.6] 

fff 

YAIP subsidized employment (%) 

83.0 

-- 

~ 

- 

79.5 

- 

~ 

- 


Average total earnings ($) 

6,255 

2,530 

3,726 *** 

[3343,4109] 

7,126 

3,891 

3,235 *** 

[2723,3747] 


Average number of quarters employed 

2.4 

1.3 

2 2 *** 

[ 1 . 0 , 1 . 2 ] 

2.7 

1.8 

Q g *H=H= 

[ 0 . 8 , 1 . 0 ] 

ff 

Average quarterly employment (%) 

52.9 

39.2 

23 7 H=H=H= 

[11.1,16.3] 

67.3 

45.6 

22 7 *** 

[18.9,24.6] 

ff 

Employment in first quarter of Year 2 (%) 

46.9 

47.3 

-0.4 

[-4.9,4.1] 

60.5 

57.3 

3.2 

[-1.3,7.8] 


Currently employed (based on survey) (%) 

47.9 

46.6 

1.3 

[-4.0,6.5] 

59.8 

56.1 

3.7 

[-1.6,9.0] 


Sample size (total = 2,605) 

788 

521 



810 

486 





SOURCES: MDRC calculations based on quarterly wage data from the National Directory of New Hires, postsecondary education data from the National Student 
Clearinghouse, and responses to the Subsidized and Transitional Employment Demonstration 12-month youth survey. 


NOTES: Program impacts were calculated separately for each subgroup, using an ordinary least squares model and adjusting for pre -random assignment 
characteristics. Impact estimates were then examined for statistically significant differences across subgroups. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a When comparing impacts between two subgroups, an H-statistic is generated. The H-statistic is used to assess whether the difference in impacts between the 
subgroups is statistically significant. Statistically significant differences between subgroups are indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 



Appendix Table D.4 


One-Year Impacts on Primary Outcomes, by Gender 





Female 




Male 


Difference 

Outcome 

Program Control 

Group Group 

Difference 

(Impact) 

90 Percent 

Confidence 

Interval 

Program Control 

Group Group 

Difference 

(Impact) 

90 Percent 

Confidence 

Interval 

Between 

Subgroup 

Impacts 3 

Education (%) 

Earned high school diploma or 

equivalency certificate (based on survey) 

8.8 

11.3 

-2.5 

[-5. 5, 0.6] 

13.4 

14.8 

-1.4 

[-5.0, 2.2] 


Enrolled in a postsecondary institution 

14.0 

13.5 

0.6 

[-2.4, 3. 6] 

11.4 

14.7 

-3.3 * 

[-6.2, -0.3] 


Enrolled in 4-year college 

5.5 

4.3 

1.2 

[-0.7, 3.2] 

3.6 

5.1 

-1.5 

[-3. 4, 0.3] 

f 

Enrolled in 2-year college 

8.5 

9.4 

-0.9 

[-3.4, 1.7] 

7.9 

9.9 

-2.0 

[-4.6, 0.6] 


Employment and earnings 

Employment (%) 

95.2 

70.3 

25.0 *** 

[22.1,27.9] 

95.0 

61.8 

2 *** 

[30.0,36.4] 

fff 

YA1P subsidized employment (%) 

Average total earnings ($) 

79.8 

6,485 

3,108 

3,377 *** 

[2975,3778] 

81.4 

6,896 

3,452 

3,444 *** 

[2943,3945] 


Average number of quarters employed 

2.6 

1.7 

Q g H=H=H= 

[0.8, 1.0] 

2.6 

1.5 

J J H=H=H= 

[1.0, 1.2] 

ff 

Average quarterly employment (%) 

64.0 

42.3 

2i 7 *H=H= 

[19.0,24.4] 

64.3 

37.2 

27 1 *H=H= 

[24.3,30.0] 

ff 

Employment in first quarter of year two (%) 

55.2 

54.8 

0.4 

[-4.0, 4.9] 

53.3 

50.0 

3.3 

[-1.2, 7.8] 


Currently employed (based on survey) (%) 

53.5 

50.1 

3.4 

[-1.7, 8.5] 

54.1 

52.9 

1.2 

[-4.0, 6.5] 


Sample size (total = 2,669) 

833 

526 



799 

511 





SOURCES: MDRC calculations based on quarterly wage data from the National Directory of New Hires, postsecondary education data from the National 
Student Clearinghouse, and responses to the Subsidized and Transitional Employment Demonstration 12-month youth survey. 

NOTES: Program impacts were calculated separately for each subgroup, using an ordinary least squares model and adjusting for pre -random assignment 
characteristics. Impact estimates were then examined for statistically significant differences across subgroups. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a When comparing impacts between two subgroups, an H-statistic is generated. The H-statistic is used to assess whether the difference in impacts between the 
subgroups is statistically significant. Statistically significant differences between subgroups are indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 



Appendix E 


Variation in Key Baseline Characteristics, Participation 
Outcomes, and Participant Program Experiences, by 

YAIP Location 
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As discussed in the main body of this report, while the multiple YAIP locations generally offered the 
same services, there were some differences in target populations, service methodologies, and other factors 
that may moderate program impacts. The effect of provider approach on program impacts will be consid- 
ered in a later report, but some preliminary analysis regarding variation by YAIP location is provided here 
as it may be of interest to readers of this report. These initial results present the distribution of provider 
locations for a variety of measures in the form of a vertical dot plot showing the values for the individual 
(anonymized) provider location in relation to the median value among the 13 provider locations that were 
part of the study. 

Appendix Figure E. 1 shows the variation in the baseline characteristics of study sample members 
by location. The largest variation by location in terms of baseline characteristics is race/ethnicity — while 
all locations served a diverse set of young people, some had greater numbers of Hispanic sample mem- 
bers while others had more Black, non-Hispanic sample members. In contrast, most locations’ sample 
members were similar in terms of age and educational attainment, consistent with YAIP’s overall target 
population. 

Appendix Figure E.2 shows variation in terms of several participation milestones. The provider 
locations had fairly similar percentages for the early milestones (attended orientation, worked in an 
internship) but had more variation for the later outcomes such as completed internship or engagement in a 
program outcome. 

Appendix Figure E.3 shows the assessment of a portion of YAIP participants regarding various 
aspects of their program experience. While there was cross-location agreement on several aspects of the 
program, the participants varied considerably across locations as to whether they thought their participa- 
tion was helpful in preparing them for future employment. 
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Appendix Figure E.l 

Characteristics of Sample Members at Baseline, by Location 


100 

90 

80 


Location 


a. 

S 

a 


Median 



19-21 years 
old 


Male 


Black Hispanic 


No Ever Ever arrested 

HSD/GED employed 
certificate 


SOURCES: Calculations based on data from MDRC's random assignment system and the YAIP management 
information system. 

NOTES: Each dot in a column represents one of the 13 YAIP locations that participated in the evaluation. The 
blue diamond represents the median value across locations. 

HSD = high school diploma. 

GED = General Educational Development. 
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Appendix Figure E.2 


One-Year Participation in YAIP Subsidized Internships and Services 
Among Program Group Members, by Location 


100 


90 


a. 

3 

o 


d 

u 

OX) 

o 

u 

a 


80 


70 


60 


50 


e 

a> 

w 

s- 

a> 

Oh 


40 


30 


20 



♦ 

1 • 


• 


— Location 

• • s 



♦ — Median 

• A 

i : 


10 


Attended 

Worked in 

Completed 

Engaged in 

Engaged in 

orientation 

internship 

internship 

outcome 

outcome 




placement at 

placement 




end of 

at 9-month 




internship 3 

follow-up 3 


SOURCES: Calculations based on data from MDRC's random assignment system and the YAIP management 
information system. 

NOTES: Each dot in a column represents 1 of the 13 YAIP locations that participated in the evaluation. The blue 
diamond represents the median value across locations. 

a Measure is calculated among program group members who ever worked in an internship. 
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Percent of YAIP participants 


Appendix Figure E.3 


YAIP Participants with a Favorable Impression of Internship Support and Preparation 

for Future Employment, by Location 


100 
90 
80 
70 
60 
50 
40 
30 
20 
10 
0 

Relationships at Supervisor support 11 Soft skills Preparation for future 

work 3 development 3 employment 11 

Sample size = 570 

SOURCE: MDRC calculations from the Subsidized and Transitional Employment Demonstration youth 
participant questionnaire. 

NOTES: Each dot in a column represents one of the 13 YAIP locations that participated in the evaluation. 
The blue diamond represents the median value across locations. 

The four measures presented in this figure were created based on an exploratory factor analysis of a 
pool of questions. These questions asked participants about their level of agreement with a particular 
statement on a scale of 1 to 7, where 1 indicates strong disagreement and 7 indicates strong agreement. 
Based on the results of the factor analysis, questions were grouped into factors and a mean score was 
calculated across the questions included in a particular factor; the percentages presented above represent 
the proportion of questionnaire respondents who averaged a score of 6 or higher on the questions in that 
factor, indicating a high level of satisfaction with their program experiences in that area. 

Questionnaires were administered to participants during weekly educational workshops at each 
provider, when many participants would be available at once. Consequently, the responses obtained are 
from participants who attended educational workshops and are therefore likely to have been more 
motivated and engaged than the full sample of program participants. For this reason, the results presented 
in this figure are not necessarily representative of all participant experiences and should be interpreted with 
caution; they are likely to be more positive. 

a Measure is based on agreement with the following statements: I understand what is expected of me on 
the job ; I know whom at work to ask for help M'hen I need if, my relationships with coworkers are positive 
and supportive ; and my coworkers understand me and want me to succeed. 


! 


i 


I 


Location 


I— 


Median 
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(continued) 


Figure E.3 (continued) 


NOTES (continued): 

b Measure is based on agreement with the following statements: 1 get the support or guidance that I need 
from my supervisor, my supervisor gives me advice about how to handle situations at work ; and my supervisor 
helps me if personal issues come up that get in the way of working. 

c Measure is based on agreement with the following statements: lam learning how to work better with 
coworkers', I am learning how to cooperate better with supervisors ; and this job has helped me learn to present 
myself better at work. 

d Measure is based on agreement with the following statements: The kind of work I am doing will help me get 
a decent-paying job later, I am learning specific job skills that I will use in the future', and 1 have met people 
through this job who may help me find a job in the future. 
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Appendix F 


Control Group Members’ Future Expectations and 
Perspectives on Obstacles Preventing Goal Achievement 



THIS PAGE INTENTIONALLY LEFT BLANK 



Appendix Figure F.l 

Control Group Members’ Future Expectations 


Will I be living in the same place in one year? 


16% 

24% 

34% 

25% 


Will I be attending a regular school in one year? 


6% 

16% 

39% 

40% 


Will I be attending school and working at least 20 hours per week in one year? 


10% 


47% 


39% 


Will I be working at least 20 hours per week in one year if I am not attending school? 




Will I become pregnant or get someone pregnant within one year? 

2% 


55% 

34% 

8% 

1 


Will I be arrested within one year? 

3% 


62% 

30% 

5% 



Will 1 have a four-year college degree by age 30? 


6% 

13% 

38% 

43% 


Will I be working at least 20 hours per week by age 30? 
3% 



4% 

33% 

60% 


■ Very unlikely Unlikely Likely ■ Very likely 

SOURCE: MDRC calculations based on responses to the Subsidized and Transitional Employment 
Demonstration 12-month youth survey. 

NOTE: Sample size = 800. 

Rounding may cause slight discrepancies in calculating sums and differences. 
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Appendix Figure F.2 


Control Group Members’ Experiences with Obstacles and Beliefs About Whether 
Obstacles Prevent Them from Achieving Their Goals 


Lack of work experience 

Lack of high school diploma or 
equivalency certificate 

Unstable housing 
Family responsibilities 
Problems with basic reading or math 
Pregnancy or child care 
Mental health difficulties 
Physical or mental disability 
Criminal record 
Experience in foster care 
Alcohol or substance abuse 



18% 


10% 


29% 


64% 

74% 

82% 

63% 


ISEl 8% 84% 

5% 

H 16% 79% 

4% 

^3% 93% 

4% 

[4% 92% 

4% 

I] 4% 93% 

2% 

| 8% 90% 

1% 

|3% 96% 


■ Prevents achieving goal Does not prevent achieving goal Not dealing with obstacle 


SOURCE: MDRC calculations based on responses to the Subsidized and Transitional Employment 
Demonstration 12-month youth survey. 

NOTE: Sample size = 800. 
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